Volume Detector

Monte Carlo’s volume detector looks at the size (normalized by time) of size updates to your tables. Volume data arrives from the warehouse/lake metadata and the detector alerts you when an update to the table is either too large, too small or was unexpected based on the historical patterns.

Our models take into account a 6 week moving window of training data and remove artifacts such as up/down spikes caused by measurement issues or the table creation processes. We then calculate features from the time-series which are used in order to decide the alert threshold for the table.

In order to achieve faster model updating based on the patterns we see our volume model is re-trained with new data every 6H.

Fine Tuning the Models

Sensitivity

On the Asset page, you can change the model's sensitivity by clicking View Details, followed by the Edit threshold sensitivity button. This change will either increase or decrease the confidence interval. By default, all thresholds are set to Medium sensitivity. Learn more about sensitivity here.

Adding Custom Volume Monitoring

Want to guarantee that tables are updated at a specific clock-time? Custom Volume Rules are great for that.

Expanded Volume Monitoring for Views, External Tables, Lakes, & Transactional Databases

Users have the option to enable volume monitoring for any External Table and View, as well as any Table on Data Lakes. The collected row count data drives both the Unchanged Size and Volume Change Detectors, which start training immediately.

Users can enable this feature directly from the asset page by clicking the "Enable Row Count monitoring" button. This action opens a drawer with additional fields to complete.