Migration

Beginning in August 2025, all customers will migrate from the legacy table monitoring system to the new Table Monitors workflow. This migration unifies monitoring and notifications, simplifies setup, and improves visibility. While some behaviors differ from the legacy system, the goal is to make it easier to understand what is being monitored and why.

Advantages

The new Table Monitors workflow has several advantages compared to the old system:

• More accessible and transparent: Table Monitors follow the same pattern as other monitors, living directly in the Monitors tab. This eliminates the need to hunt through Settings or learn a separate interface.
• Simpler and easier to manage: Legacy monitoring relied on a web of monitoring rules, audience rules, and reporting filters. These relationships were often complex and confusing. Table Monitors consolidate all of this into a single, end-to-end workflow where you can scope, configure, and manage everything in one place.
• User-friendly ownership: Individual users can create and edit their own monitors without disrupting other teams’ configurations. Every monitor is easy to reference later, so you always know what’s being monitored and why.
• Encourages broader adoption: By reducing complexity and making the workflow straightforward, we expect more users to create more Table Monitors. This means greater coverage, more proactive monitoring, and ultimately higher confidence in your data.

Process

• We will automatically migrate your existing usage and notification rules into new Table Monitors.
• Each new Table Monitor will preserve the original scope and audience of the corresponding rules.
• You will receive advance notice before the migration occurs.
• After migration, you will be able to review all newly created Table Monitors and compare them to your previous configuration.
• If you identify any discrepancies, please reach out to our support team to resolve them quickly.

Changes

• Table monitoring and notifications will be managed entirely through Table Monitors.
• The Settings > Table Monitors and Settings > Notifications pages will no longer apply.
• Alerts will be routed through monitor-defined audiences only.
• The “monitored” state of a table will be determined exclusively by Table Monitors.

New Table Monitor Overview

Table Monitors in Monte Carlo provide a unified and streamlined way to monitor tables for freshness, volume, and schema anomalies. This redesign replaces the legacy system where table monitoring was configured separately through the Settings UI and dependent on usage and notification rules. Each monitor runs on an automatic hourly schedule and can detect:

• Freshness anomalies when updates are missing or delayed
• Volume anomalies when row counts or data size diverge from learned or defined thresholds
• Schema changes such as added or removed columns

Table Monitors are easy to configure, flexible in scope, and fully visible in the UI. Every Table Monitor can be created, updated, or reviewed directly within the Monitors tab, just like any other monitor.

Configuration

To create a Table Monitor:

1. Go to the Monitors tab in Monte Carlo.
2. Click Create Monitor and select Table monitor.
3. Choose a warehouse.
4. Optionally filter the scope by selecting one or more databases or schemas.
5. Define rules to include or exclude tables:
   • Table name (starts with, ends with, contains, matches pattern)
   • Table tags (has any / has all)
   • Table type (base table or view)
   • Read/write activity within a defined window (e.g., past 30 days)
6. Assign audiences to receive notifications.
7. Set the priority (e.g., P1, P2) and give the monitor a name.
8. Save the monitor.

📘

Monitors created while a specific domain is selected in the UI, will be associated to that domain. These monitors will be visible and editable to users with access, to that domain or to users without domain restrictions. They will only monitor tables included in that domain.

Once created, Monte Carlo automatically applies its default anomaly detection checks to the selected tables and begins hourly monitoring.

Permissions

Monitor Editors can now create Table Monitors. Previously, only users with the Editor role could manage table monitoring via Settings > Table Monitors.

Domain-based access control still applies:

• If a user is restricted to a specific domain, any Table Monitor they create will only apply to tables within that domain — even if the selected tag spans multiple domains.
• To monitor tables across multiple domains, a non-restricted user must create the monitor, or multiple domain-restricted users must each create monitors for their respective domains.

Visibility and alerting:

• Table Monitors are only visible and editable by users with access to the relevant domain(s).
• Alerts still route to the appropriate table owners and stakeholders within their domain.

Alerts

Each Table Monitor is responsible for generating alerts based on the tables it covers. Alerts are created as follows:

• A single anomaly may be referenced by multiple Table Monitors (if the table is shared), but each monitor will generate a separate event and alert.
• Alerts are routed to the audiences assigned to the Table Monitor.
• The Table Monitor’s priority level determines alert escalation behavior.
• All standard checks (freshness, volume, schema) are applied by default. You can override thresholds, disable checks, or define static thresholds per table if needed.

You can view all alerts triggered by a Table Monitor, review their incident details, and trace them back to the original table.

Duplicate Alerts

• If an anomaly is detected on a table, every monitor that covers that table will generate an alert.
• You may see more alerts than before, sometimes for the same underlying anomaly.
• Handling duplicates now requires choosing how to scope monitors and assign audiences.

Monitoring with Domains and Data Products

As you migrate, it’s important to understand how Table Monitors relate to Domains and Data Products. This will help you set up alerting in a way that aligns with team ownership while minimizing maintenance overhead.

Principles

• Domains represent teams. They own and manage their own monitoring and alerting.
• Data Products represent tables that are critical to stakeholders.
• Each team should manage its own monitors. If another team depends on a Data Product, they should request the owning team to monitor it.

Best Practices for Monitoring Data Products

1. Use one Table Monitor per domain per warehouse.

   • This keeps monitoring centralized and avoids alert duplication.
   • The Table Monitor is automatically scoped to the team’s domain during creation.

2. Add new Data Products to the existing Table Monitor.

   • When a new Data Product is created, the owning team can add it to their Table Monitor.
   • This ensures consistent coverage without proliferating monitors.

3. Choose the right construct for monitoring.

   • Data Products are powerful for modeling business use cases (e.g., financial reporting).
   • For upstream or ingestion teams schemas may be a better fit for scoping Table Monitors.
   • The key is to align monitoring with team ownership and what matters most.

Example

• A Finance domain creates a Table Monitor scoped to its warehouse and adds all Data Products tied to reporting.
• An Ingestion domain instead creates Table Monitors at the schema level, since their ownership is tied to source data pipelines rather than specific Data Products.

Notifications

Table Monitors use audiences to manage where alerts are sent. An audience can be a group of users or integrated systems (such as Slack or PagerDuty). Alerts are routed to each monitor’s audiences without the need for global notification rules.

Legacy notification rules are ignored for any incident covered by a Table Monitor. During the migration period, we ensure that alerts are only sent once and routed through the monitor.

Billing

• A table is considered monitored if it is included in at least one Table Monitor.
• Billing is calculated based on the number of unique monitored tables, not the number of monitors.
• If a table is no longer covered by any monitor, it will no longer generate alerts and will not count toward your billed table total.

API Changes

As part of the changes introduced for Table Monitors, the following GraphQL operations are no longer supported:

• UpdateMonitoredTableRuleList,UpdateMonitoredTableRuleListAsync, AddMonitoredTableRuleAsync: monitored table rules are fully replaced by Table Monitors and can no longer be configured independently.
• CreateOrUpdateAudienceRoutingRule: The operation can be used to create routing rules for jobs, but no longer accepts parameters to select specific alert types or assets. In particular, the following properties are no longer accepted:
  • digestSettings, tableRegex, projectNames, projectMcons, datasetIds, fullTableIds, tableMcons, excludeFullTableIds, excludeTableMcons, excludeProjectNames, excludeProjectMcons, excludeDatasetIds, excludeAssetsMcons, excludeTagKeys, anomalyTypes, incidentSubTypes.

To create new Table Monitors or editing existing ones, you can use the following new GraphQL operation: createOrUpdateTableMonitor.