Generating MC monitors from dbt tests

Overview

If you already maintain dbt tests, you don't have to rewrite them as monitors by hand. Point the Monte Carlo CLI (0.44.0+) at your dbt manifest.json and it generates MaC YAML — one monitor per dbt test — that you can commit to your project and deploy like any other monitor.

This is one of two ways to drive Monte Carlo monitors from dbt; you can also embed monitors directly in a model's meta block in schema.yml — see Embedded in dbt schema.yml.

Generating monitors from dbt tests

Point the CLI at your dbt manifest.json and it produces MaC YAML you can add to your project:

montecarlo monitors generate-from-dbt-tests \
  --dbt-manifest /path/to/dbt/manifest.json \
  --output-path /path/to/mc_monitor_config.yml

You can filter by test type: any of the standard four (unique, not_null, accepted_values, relationships), a custom generic test type, or SINGULAR for singular tests. Use --label to attach audiences to all generated monitors (label is equivalent to audience):

montecarlo monitors generate-from-dbt-tests \
  --dbt-manifest /path/to/dbt/manifest.json \
  --output-path /path/to/mc_monitor_config.yml \
  --test-type SINGULAR \
  --test-type not_null \
  --label foo \
  --label bar

📘

Make sure to apply the changes to your Monte Carlo project

The montecarlo monitors generate-from-dbt-tests command creates a YAML config file that will create the corresponding monitors for the dbt tests when the monitor config is applied. It does not immediately apply the config. Make sure to run the montecarlo monitors apply command described above to actually create the monitors.

The generated output will look like this, with an entry for each dbt test:

montecarlo:
  custom_sql:
    - name: test.dbt_project.not_null_table_name_field_name
      description: test.dbt_project.not_null_table_name_field_name
      sql: |
        select field_name
        from dbt_project.prod.table_name
        where field_name is null
      schedule:
        type: fixed
        start_time: '2023-03-30T17:33:52.897469+00:00'
        interval_minutes: 720
      alert_conditions:
        - type: threshold
          operator: GT
          threshold_value: 0
      priority: P2
      tags:
        - name: source
          value: dbt_test
      data_quality_dimension: COMPLETENESS
      audiences:
        - bar
        - foo

📘

Older CLI versions may generate comparisons instead of alert_conditions. Both are accepted, but alert_conditions is the current field name. See Custom SQL Monitor for the full reference.