We believe that effective monitoring solutions must minimize operational and compute costs associated with creating monitors, while maximizing the number of relevant issues detected. We accomplish this by deploying 3 types of monitors:
- Full coverage ML-driven detection
- Opt-in coverage ML-driven detection
- Custom Rule-based detection
For more information on how our machine learning driven detection works, please see our Anomaly Detection section.
|Monitor Type||Issues Detected||How to Enable|
|Automatic ML-driven detection||Freshness: MC monitors how often each table in your environment is updated, and alerts you if there is a delay|
Volume: MC monitors how much data is added/ removed/ updated for each table with each update, and alerts you if tables grow or shrink unexpectedly
Schema: MC monitors all schema changes in your environment, and alerts you of added, removed or updated fields and deleted tables
|No action needed. These are automatically enabled for all tables which MC has access to. For views, only schema detection will apply.|
|Opt-in ML-driven detection||Field Health: MC calculates and monitors multiple metrics and statistics (see full list here), and alerts you if there are substantial changes in those metrics and statistics|
Dimension Tracking: MC monitors the frequency of field values (best for low-cardinality fields) and alerts you of unexpected changes in the distribution
JSON Schema: MC monitors fields with nested JSON values for changes in the structure, and alerts you if that structure changes
|Navigate to the Monitors tab and select the type of monitor you want to enable. Follow the flow to opt-in your important data tables, views, or specific columns.|
|Custom Rule-based detection||SQL Rules: MC makes it easy for data teams to define custom rules using SQL statements to check for specific conditions, and alerts you when those conditions are breached|
SLOs: If the automatic out-of-the-box ML based monitoring isn't enough for you use cases, you can set up SLOs which allow you to set the threshold for freshness and volume, and MC will alert you once that threshold is breached. You have the option to set up Freshness SLOs or Volume SLOs.
Ingestion Vaildation: Ingestion Validation is designed to check that data coming from external partners is landing in your warehouse on-time, and in the size and quality that you expect. Monte Carlo automatically monitors these tables for freshness and volume anomalies.
|Navigate to the Monitors tab and select the type of Rule you would like to set up.|
Updated about 1 month ago