Detectors Overview

Monte Carlo supports two main methods of detecting anomalies:

1. Automatic, machine learning based detections: these detections are made based on automatic and changing thresholds determined by our models based on historical data we have received from your data source.
2. User set detections: these detections are made based on thresholds set by the user. They are either a hard threshold, for example: create an anomaly when my table hasn't updated in the past 10 hours, or they are a percentage based threshold, for example: create an anomaly when my table has loaded less than 25% of the average loaded rows from the past 7 days.

The below sections will go into how we think about our automatic, machine learning based detectors, and various ways you can interpret and interact with them.

Out of the Box Automatic Detectors

As part of its data observability platform MC provides automatic metadata based anomaly detection on all your tables. Note that these ML models will only be active for tables where metadata is available and where the table’s data follow some basic criteria for number of samples, period of the signal and a few more.

How do they work?

Monte Carlo has created a mix of time-series models with changes we introduce to make them work better with our specific problem and statistical models. Each type of detector model is unique and built specifically around that type of data (freshness, volume, unchanged size, etc.)

In general our models set the thresholds based on:

1. Pattern type classification (I.E streaming_table / weekend_pattern / multimodal_update_pattern / etc.)
2. Historical trends and value distributions
3. State change detection
4. Some rules learned from user feedback over time
   On top of all this sits a model which is a mix of of time-series models with changes we introduce to make them work better with our specific problem and statistical models based on historical distributions and probabilities.

A note to make is that since these automated detectors run on ALL of your tables, we have made the choice to tune our models more towards precision rather than recall in order to avoid alert fatigue. For cases where a very strict threshold is required we usually suggest using our SLOs.

Another important note is that while we use large validation sets from 1000s of user feedback we have collected, our models train on your data alone and a specific model is trained for each of your tables. This allows us to learn from our large user base while not using your data to train models for other clients and giving each table the highest level of accuracy based on its own time series patterns.

These automatic detectors are available for freshness data (when was the table updated) and volume data (when and how much data was added/removed from the table). For more details on each detector, please see below:

• Freshness
• Volume
• Unchanged Size (volume with freshness)

Detector Statuses

Monte Carlo's automatic detectors can exist in one of three states:

• Training
  This status usually appears during the first 2 weeks after onboarding or detecting a new table. It does not mean that the detector will not be active but rather that the anomalies might not be perfect since the model has had very little data to learn from.
• Active
  The detector is live and working as expected.
• Inactive
  The detector is not active due to the training data not passing some minimum requirements for our models.
  While the detector is “inactive” it does not mean it will never alert but rather that it goes into a sort of low-confidence mode under which we will only alert you on very significant events.

You can usually know if the detector is active during training based on whether or not you see a threshold in the MC app catalog page.