Improved
Improvement to row counts for Custom SQL & Validation monitors on data lakes
about 6 hours ago
Custom SQL and Validation monitors on data lake engines now report the true total row count, independent of the sampling limit.
Previously, with sampling enabled on lake engines (Databricks, Starburst, Athena, Hive, Presto, Spark, Dremio) and when monitoring for a count of rows, the reported count was capped at the sampling limit.
- On by default for monitors created from August 17, 2026 onward
- For monitors created before that date, reach out via our Support Agent or your account representative to enable it. On the first run after enabling, the count corrects from the cap to its true value — expect one extra alert on manual thresholds, a run or two of alerts on automated thresholds while they re-baseline, and a one-time step change in breach-count charts and DQ dashboards
- Sample rows are unchanged (still capped at 500, or 5,000 where the increased limit is enabled). Only the total row count reported has changed.
Learn more here: https://docs.getmontecarlo.com/docs/creating-sql-rules#note-for-data-lakes