Table
Monte Carlo makes it easy to instantly deploy broad monitoring of your data pipelines, with no manual configuration. It learns the normal patterns of updates, size changes, and size growth, and alerts when those patterns are violated. This is ideal for detecting breakages and stoppages in the flow of data.
For regularly updated tables, Monte Carlo can typically generate thresholds after 7 days of observing a table, and thresholds will be mature by 14 days. At most, monitors will incorporate a 6-week rolling window of training data to determine thresholds, including handling of weekly seasonality.
The sensitivity of thresholds can be adjusted between High, Medium, and Low. Users may also choose a manual threshold, instead of relying on thresholds generated by machine learning. Learn more about sensitivity settings.
For most integrations, these monitors start working immediately without user configuration and use metadata instead of directly querying the table. Table Monitors include:
- Freshness: alerts to unusually long periods of time between updates or changes in size (row count or bytes)
- Volume: alerts to unusual changes in size
- Schema change: alerts to changes in the schema of the table (no machine learning involved here)
Table Monitor Migration
We are currently migrating all customers from the legacy table monitoring system to the new Table Monitors workflow. The goal is to simplify how monitoring and notifications are defined and reduce confusion from overlapping systems.
Process
- We will automatically migrate your existing usage and notification rules into new Table Monitors.
- Each new Table Monitor will preserve the original scope and audience of the corresponding rules.
- You will receive advance notice before the migration occurs.
- After migration, you will be able to review all newly created Table Monitors and compare them to your previous configuration.
- If you identify any discrepancies, you can reach out to our support team to resolve them quickly.
Changes
- Table monitoring and notifications will be managed entirely through Table Monitors.
- The Settings > Table Monitors and Settings > Notifications pages will no longer apply.
- Alerts will be routed through monitor-defined audiences only.
- The “monitored” state of a table will be determined exclusively by Table Monitors.
This migration is designed to preserve alert coverage while simplifying setup and making it easier to audit what’s being monitored and why.
New Table Monitor Overview
Table Monitors in Monte Carlo provide a unified and streamlined way to monitor tables for freshness, volume, and schema anomalies. This redesign replaces the legacy system where table monitoring was configured separately through the Settings UI and dependent on usage and notification rules. Each monitor runs on an automatic hourly schedule and can detect:
- Freshness anomalies when updates are missing or delayed
- Volume anomalies when row counts or data size diverge from learned or defined thresholds
- Schema changes such as added or removed columns
Table Monitors are easy to configure, flexible in scope, and fully visible in the UI. Every Table Monitor can be created, updated, or reviewed directly within the Monitors tab, just like any other monitor.
Configuration
To create a Table Monitor:
- Go to the Monitors tab in Monte Carlo.
- Click Create Monitor and select Table monitor.
- Choose a warehouse.
- Optionally filter the scope by selecting one or more databases or schemas.
- Define rules to include or exclude tables:
- Table name (starts with, ends with, contains, matches pattern)
- Table tags (has any / has all)
- Table type (base table or view)
- Read/write activity within a defined window (e.g., past 30 days)
- Domain (select one or more)
- Assign audiences to receive notifications.
- Set the priority (e.g., P1, P2) and give the monitor a name.
- Save the monitor.
Once created, Monte Carlo automatically applies its default anomaly detection checks to the selected tables and begins hourly monitoring.
Alerts
Each Table Monitor is responsible for generating alerts based on the tables it covers. Alerts are created as follows:
- A single anomaly may be referenced by multiple Table Monitors (if the table is shared), but each monitor will generate a separate event and alert.
- Alerts are routed to the audiences assigned to the Table Monitor.
- The Table Monitor’s priority level determines alert escalation behavior.
- All standard checks (freshness, volume, schema) are applied by default. You can override thresholds, disable checks, or define static thresholds per table if needed.
You can view all alerts triggered by a Table Monitor, review their incident details, and trace them back to the original table.
Notifications
Table Monitors use audiences to manage where alerts are sent. An audience can be a group of users or integrated systems (such as Slack or PagerDuty). Alerts are routed to each monitor’s audiences without the need for global notification rules.
Legacy notification rules are ignored for any incident covered by a Table Monitor. During the migration period, we ensure that alerts are only sent once and routed through the monitor.
Billing
- A table is considered monitored if it is included in at least one Table Monitor.
- Billing is calculated based on the number of unique monitored tables, not the number of monitors.
- If a table is no longer covered by any monitor, it will no longer generate alerts and will not count toward your billed table total.
Updated 2 days ago