8.9.21 Changelog

The MC engineering team is running a hackathon this week so we'll have fewer updates to share next week.

What's new

  • Filter notifications with table tags: To support more advanced notification filters, you can now use table tags to filter which notifications go to which Slack channels, emails, etc.

  • Data Lake throughput anomaly detection: To provide the same observability Monte Carlo offers around volume changes in data warehouses, we shipped support for throughput monitoring in Data Lakes which will alert you to abnormal volume changes and unchanged size anomalies

  • More UI usability improvements:

    • Select only to filter by a single monitor type in the Custom Monitors table
    • Update table name formatting across the application using the following standard format [data warehouse name] database_name:schema_name.table_name
    • Open incidents in new tab from Pipelines graph incidents
    • Edit date time using text input in addition to UI date selector selector component
    • Add search filter to field selector when enabling Field Health monitoring
    • Select all fields in the Field Health monitor flow by default

Improvements and Fixes

  • Improved handling of low distinctness numeric field health monitors: We now provide better support for catching numerical anomalies in tables with very low distinct values
  • Faster anomaly detection training period: To reduce the training time for anomaly detection models, we now pull 2 weeks of historical data when a monitor is deployed
  • Select Data Collector Region during warehouse setup: When onboarding a new data warehouse, you will now be able to select your data collector region in the app

What's next

  • SQL Rule Snooze: To minimize unwanted notifications during an investigation, we'll soon support the ability to snooze SQL Rule notifications for preset time period
  • Workspaces: To support large organizations with multiple teams, we're working on a workspace model that will allow you to create collections of projects and schemas that will filter what the user sees in the UI
  • Monitors as code: A highly requested feature from our roadmap sessions was to configure Monte Carlo as a part of the code release process - this integration will support YAML based configs to deploy custom monitors within Monte Carlo