JSON Schema Monitor

Overview

Detect structural changes in a JSON column — new keys appearing, existing keys vanishing, or value types shifting. Point it at any semi-structured column (JSON, VARIANT, SUPER, etc.) to catch upstream schema drift before it breaks downstream consumers.

📘
Reference scope
This page covers MaC YAML configuration. For how JSON schema monitors work, see JSON Schema Monitors.

MaC key: json_schema.

Quick Start

montecarlo:
  json_schema:
    - name: raw_events_payload_schema
      description: Detect schema changes in raw event payloads
      table: analytics:events.raw_events
      field: payload
      timestamp_field: received_at
      lookback_days: 2
      schedule:
        type: fixed
        interval_minutes: 720
      domains:
        - my-domain

Configuration

table — fully qualified table name

string · required

Format: database:schema.table.

table: analytics:events.raw_events

field — JSON column to monitor

string · required

Name of the JSON column to monitor for schema changes.

field: payload

timestamp_field — timestamp column for lookback window

string · optional

Timestamp column used to scope the lookback window. Without this, the monitor scans all available rows.

timestamp_field: received_at

timestamp_field_expression — SQL expression for timestamp

string · optional

SQL expression that evaluates to a timestamp, used instead of timestamp_field when the column needs transformation.

timestamp_field_expression: "TO_TIMESTAMP(epoch_seconds)"

lookback_days — days of data to scan

integer · optional

Number of days to look back when scanning data. Recommended range: 0--7. Only effective when timestamp_field is set.

lookback_days: 2

where_condition — SQL WHERE filter

string · optional

SQL WHERE clause (without the WHERE keyword) to filter rows before analysis.

where_condition: "event_type = 'purchase'"

unnest_field — nested JSON array to unnest

string · optional

Name of a nested JSON array field to unnest before schema analysis. Use when the JSON column contains an array of objects and you want to monitor the schema of the array elements.

unnest_field: items

aggregation_time_interval — time bucket granularity

enum · optional

Accepted values: hour · day · week · month

aggregation_time_interval: day

schedule — execution schedule

object · optional · default: system-managed schedule

Controls when the monitor runs. JSON schema monitors do not support crontab-based scheduling.

schedule:
  type: fixed
  interval_minutes: 720

Properties

type — schedule type

enum · optional · default: fixed

Accepted values: fixed · dynamic

Crontab and manual are not supported. loose is deprecated — the backend rejects it with "Loose schedules are deprecated, use fixed instead."

type: fixed

interval_minutes — run interval

integer · optional

Run interval for fixed schedules.

interval_minutes: 720

start_time — schedule start time

string · optional

ISO 8601 format.

start_time: "2024-01-01T06:00:00Z"

timezone — schedule timezone

string · optional

timezone: America/New_York

dynamic_schedule_tables — tables that trigger the monitor

array of strings · required when type is dynamic (unless dynamic_schedule_jobs is set)

dynamic_schedule_tables:
  - raw:api.webhook_responses

dynamic_schedule_jobs — jobs that trigger the monitor

array of objects · optional

Property	Type	Required	Description
`job_type`	string	yes	`AdfJob` · `AirflowDag` · `DatabricksJob` · `DbtJob`
`job_name`	string	yes	Name of the job
`project_name`	string	yes	Project or workspace containing the job
`task_name`	string	no	Specific task within the job
`mcon`	string	no	MCON identifier for the job

dynamic_schedule_jobs:
  - job_type: AirflowDag
    job_name: raw_events_pipeline
    project_name: data-platform

min_interval_minutes — minimum interval between runs

integer · optional

Minimum interval between runs for dynamic schedules.

min_interval_minutes: 60

domains — domain for this monitor

array of strings (exactly one entry) · required on all accounts created after January 2025

Set default_domain in montecarlo.yml to avoid repeating it on every monitor.

domains:
  - my-domain

warehouse — which warehouse to use

string · optional · required if multiple warehouses

Warehouse UUID or name. Overrides default_resource from montecarlo.yml.

warehouse: prod-snowflake

name — unique identifier within the namespace

string · required

Required for monitors created after Jan 29, 2024 (existing monitors keep working). Changing the name creates a new monitor and deletes the old one.

name: raw_events_schema_check

description — monitor description

string · required

Max 512 characters.

description: Detect schema changes in raw event payloads

notes — internal notes

string · optional

Visible in the Monte Carlo UI. Not included in notifications.

notes: Owned by the data platform team.

audiences — notification channels

array of strings · optional

Audience names linking this monitor to channels defined in Notifications as Code.

audiences:
  - data-engineering-alerts
  - api-integration-alerts

failure_audiences — run-failure notification channels

array of strings · optional

Separate audiences for run-failure notifications. Falls back to audiences if not set.

failure_audiences:
  - oncall-alerts

notify_run_failure — notify on query failure

boolean · optional

Notify when the monitor query itself fails.

notify_run_failure: true

severity — severity level

string · optional

Severity level for incidents created by this monitor.

severity: SEV-2

priority — incident priority level

enum · optional

Accepted values: P1 · P2 · P3 · P4 · P5

priority: P3

connection_name — named connection override

string · optional

Overrides the default warehouse connection.

connection_name: analytics-read-only

disable_look_back_bootstrap — skip initial bootstrap

boolean · optional

Skip the initial historical data bootstrap when the monitor is first created.

disable_look_back_bootstrap: true

tags — key-value pairs for organizing monitors

array of objects · optional

Property	Type	Required	Description
`name`	string	yes	Tag key
`value`	string	no	Tag value

tags:
  - name: team
    value: data-platform
  - name: source
    value: api

data_quality_dimension — data quality category

enum · optional

Accepted values: ACCURACY · COMPLETENESS · CONSISTENCY · TIMELINESS · UNIQUENESS · VALIDITY

data_quality_dimension: CONSISTENCY

is_draft — create as draft without activating

boolean · optional · default: false

Creates the monitor in a paused state. Omitting this on a later update resets to false (active) due to PUT semantics — always include it if you want the monitor to stay in draft.

is_draft: true

uuid — update an existing monitor

string · optional

Include the UUID of an existing monitor to update it instead of creating a new one.

uuid: 0dae7702-0950-45c7-909c-7e183bddca19

Deprecated fields

Field	Use instead
`resource`	`warehouse`
`domain`	`domains`
`domain_uuids`	`domains`
`labels`	`audiences`
`notify_rule_run_failure`	`notify_run_failure`

📘
API-only fields
Some fields visible in the API or JSON Schema (skip_reset, fail_on_reset, select_expressions) are present in the schema but silently stripped during MaC YAML processing. They have no effect and should not be included.

Examples

Basic JSON schema monitoring

Monitors a JSON column for structural changes, looking back 2 days from each run.

montecarlo:
  json_schema:
    - name: raw_events_payload_schema
      table: analytics:events.raw_events
      field: payload
      timestamp_field: received_at
      lookback_days: 2
      description: Detect schema changes in raw event payloads
      schedule:
        type: fixed
        interval_minutes: 720
      audiences:
        - data-engineering-alerts
      priority: P3
      data_quality_dimension: CONSISTENCY
      tags:
        - name: team
          value: data-platform
      domains:
        - my-domain

Monitoring nested JSON arrays

Uses unnest_field to monitor the schema of objects inside a nested JSON array.

montecarlo:
  json_schema:
    - name: webhook_items_schema
      table: raw:api.webhook_responses
      field: response_body
      unnest_field: items
      timestamp_field: created_at
      lookback_days: 3
      description: Monitor schema of items array in webhook response bodies
      schedule:
        type: dynamic
        dynamic_schedule_tables:
          - raw:api.webhook_responses
      audiences:
        - api-integration-alerts
      priority: P2
      domains:
        - my-domain

Filtered monitoring with WHERE clause

Monitors JSON schema only for a specific event type.

montecarlo:
  json_schema:
    - name: purchase_properties_schema
      table: analytics:events.raw_events
      field: properties
      where_condition: "event_type = 'purchase'"
      timestamp_field: event_time
      lookback_days: 1
      aggregation_time_interval: day
      description: Track schema stability of purchase event properties
      schedule:
        type: fixed
        interval_minutes: 1440
      priority: P2
      domains:
        - my-domain

Troubleshooting

Timestamp and lookback

Omitting timestamp_field when using lookback_days. Without a timestamp column, lookback_days has no effect and the monitor scans all rows.
Setting a large lookback_days. Values outside the recommended 0--7 range are accepted at apply time but a wide window increases scan cost; keep it small unless you have a reason not to.

Scheduling

Using crontab scheduling. JSON schema monitors do not support interval_crontab or interval_crontab_day_operator. Use interval_minutes for fixed schedules.
Using manual or loose schedule type. JSON schema monitors support fixed and dynamic schedule types only. loose is deprecated and rejected by the backend.

Data source and fields

Using data_source. JSON schema monitors do not support a data_source block — the backend rejects it with "Data source is not supported for this monitor type." Specify the table with the top-level table field instead.
Wrong tags format. Tags must be objects with name and optional value keys. Writing tags: ["my-tag"] fails validation.

Updates and deprecated fields

Forgetting PUT semantics on updates. When updating a monitor by including uuid, every field you omit reverts to its default — it is not left unchanged. Always specify the complete desired configuration.
Using resource instead of warehouse. The resource field still works but is deprecated. Use warehouse in all new configurations.