Monitors Overview

We believe that effective monitoring solutions must minimize operational and compute costs associated with creating monitors, while maximizing the number of relevant issues detected. We accomplish this by deploying 3 types of monitors:

  • Full coverage ML-driven detection
  • Opt-in coverage ML-driven detection
  • Custom Rule-based detection

For more information on how our machine learning driven detection works, please see our Anomaly Detection section.

Monitor TypeIssues DetectedHow to Enable
Automatic ML-driven detectionFreshness: MC monitors how often each table in your environment is updated, and alerts you if there is a delay

Volume: MC monitors how much data is added/ removed/ updated for each table with each update, and alerts you if tables grow or shrink unexpectedly

Schema: MC monitors all schema changes in your environment, and alerts you of added, removed or updated fields and deleted tables
No action needed. These are automatically enabled for all tables which MC has access to. For views, only schema detection will apply.
Opt-in ML-driven detectionField Health: MC calculates and monitors multiple metrics and statistics (see full list here), and alerts you if there are substantial changes in those metrics and statistics

Dimension Tracking: MC monitors the frequency of field values (best for low-cardinality fields) and alerts you of unexpected changes in the distribution

JSON Schema: MC monitors fields with nested JSON values for changes in the structure, and alerts you if that structure changes
Navigate to the Monitors tab and select the type of monitor you want to enable. Follow the flow to opt-in your important data tables, views, or specific columns.
Custom Rule-based detectionSQL Rules: MC makes it easy for data teams to define custom rules using SQL statements to check for specific conditions, and alerts you when those conditions are breached

Freshness and Volume Rules: If the automatic out-of-the-box ML based monitoring isn't enough for you use cases, you can set up Rules which allow you to set the threshold for freshness and volume, and MC will alert you once that threshold is breached. You have the option to set up Freshness Rules or Volume Rules.
Navigate to the Monitors tab and select the type of Rule you would like to set up.