Monitors Overview
Monte Carlo offers a variety of monitors so that data teams can:
- Instantly deploy broad monitoring of your data pipelines, with no manual configuration. These monitors use metadata whenever possible so that compute costs are minimal.
- Strategically deploy deep quality monitoring on the tables that are more important. These monitors directly query the data, to alert to shifts in the data's profile, failed validations, and more.
Monitor Categories
Monitors in Monte Carlo fall into 4 categories. In most cases, users can choose between automated or manual thresholds. For more information on our machine learning detection, please see our Anomaly Detection section.
| Category | Description |
|---|---|
| Table | Monte Carlo learns the normal patterns of updates, size changes, and growth for any given table, and alerts if those patterns are violated. Ideal for detecting breakages and stoppages in the flow of data. For most integrations, these monitors start working immediately without user configuration and use metadata instead of querying the table. Monitors: Freshness, Volume, and Schema Change |
| Metric | Monte Carlo learns the behavior of a metric and alerts when those patterns are violated. These can be statistical metrics of fields (like unique % or mean) or custom metrics defined by the user. Ideal for detecting breakages, outliers, "unknown unknowns," and anomalies within specific segments of data. See all available metrics. Monitors: Metric Monitors |
| Validation | Users can choose from templates or write their own SQL to check specific qualities of the data. Ideal for row-by-row validations, business-specific logic, or comparing metrics across multiple sources. See all available conditions. Monitors: Validation Monitors, Custom SQL and Comparisons |
| Job | Users can track runtime of queries and be alerted to outliers and queries that aren't scaling well. Ideal for ensuring that pipelines remain reliable and that costs aren't spiraling out of control. In addition, Monte Carlo can ingest alerts from tools like Airflow, dbt, Azure Data Factory, and Databricks to provide a more comprehensive view of what has occurred during an incident. Monitors: Query Performance |
Updated about 2 months ago