Guides
HomeChangelogStatus PageAPI ExplorerAPI ReferenceCLI ReferenceBlogRequest a Demo

Insights (deprecated)

Access Monte Carlo generated metadata

Suggest Edits

🚧

Insights deprecation

We're sunsetting the legacy Insights system in favor of Data exports. Here's what to expect:

  • The Insights tab will be removed from the UI
  • The “Monitored Tables” insight has been moved to Settings → Table monitors
  • Full deprecation will be July 1, 2025. If you have concerns about losing specific insight reports, please reach out to your customer success manager or [email protected].

Migrating from Insights to Data Exports

Legacy InsightData Export(s)Renamed columnsGaps / no-longer-supported / must-derive
EventsEventsMC_INCIDENT_ID → ALERT_ID
TYPE → EVENT_TYPE, CREATED_AT, ANOMALOUS_DATA_TIME → ANOMALY_TIMESTAMP,
ACTUAL → ACTUAL_VALUE, DOMAINS
MIN_EXPECTED → LOWER_THRESHOLD_VALUE,
MAX_EXPECTED → UPPER_THRESHOLD_VALUE		Per-alert status metrics (FIRST_STATUS*, LAST_STATUS*, HOURS_TO_* → now in Alerts);
Incident HistoryAlertsINCIDENT_ID → ALERT_ID, INCIDENT_CREATED_TIMESTAMP → CREATED_AT, LAST_FEEDBACK → STATUSAsset & event roll-ups (N_EVENTS, N_TABLES, N_KEY_ASSETS, importance aggregates); Can be derived.
Misconfigured MonitorsMonitorsMONITOR_UUID → MONITOR_ID, MONITOR_STATUS → STATUS, CREATED_ON → CREATED_ATSUGGESTION text, AGG_TYPE; old filter flags for “Effective/Warming Up/Paused” need to be rebuilt via SQL conditions.
Key AssetsAssets / DeprecatedIdentification: RESOURCE_TYPE, FULL_TABLE_ID → FULL_ASSET_NAMEAll usage-frequency metrics (READS, WRITES, distinct-user counts, periodicity, last-read/write timestamps, size stats). Must be calculated from warehouse logs or INFORMATION_SCHEMA.
Coverage OverviewMonitors x AssetsBoolean coverage flags can be recreated by left-joining and flagging monitor presence.No pre-computed booleans; must derive SCHEMA_MONITOR, FRESHNESS_MONITOR, etc., via SQL.
Notifications by Custom MonitorEvents x Monitors—Can be computed by Events and Monitors.
Table Cleanup SuggestionsDeprecated—Dormancy metrics (LAST_READ, DAYS_SINCE_LAST_READ, TOTAL_ROW_COUNT, TOTAL_BYTE_COUNT, …) not shipped; rebuild from warehouse metadata.
Field-level Cleanup SuggestionsDeprecated—Field-granular lineage / usage counts and %_REMOVABLE logic are not exported.
Field Importance ScoresDeprecated—Field-level importance is available in the UI/API.
Field Health SuggestionsDeprecatedField-level importance is available in the UI/API.
BI Dashboard AnalyticsDeprecated—Looker dashboard view counts and stale-dashboard heuristics have been deprecated. This information is available still in the Monte Carlo UI/API and natively in Looker.
Rule ResultsDeprecated—Not available; tracked on the roadmap.

We offer multiple mechanisms to access the metadata Monte Carlo collects to support a range of analytical and tracking use cases. Many of our customers today access this data to define and track SLI performance, monitor data asset usage, determine the most important (or least important) data assets, and more.

The following document outlines the different options you have for accessing this data and provides a summary of the available inventory.

Mechanisms

Dashboard

Users can easily download CSV reports right from the UI. Navigate to the "Dashboard" tab and click to download the CSV reports:

571

Insight Reports

Snowflake Data Marketplace

For Snowflake users, we offer the ability to get our Insights via Snowflake's Data Sharing capabilities. This allows users to get their Insight reports synced directly into their Snowflake instance and use them in queries/dashboards.

This is an opt-in feature, and is available in the following Snowflake data warehouse regions:

  • AWS:
    • us-east-1
    • us-east-2
    • us-west-2
    • ca-central-1
    • eu-west-1
    • eu-central-1
    • ap-northeast-1
    • ap-southeast-2
  • Azure:
    • eastus2
    • centralus
    • westeurope
    • australiaeast
    • canadacentral
  • GCP:
    • us-central1
    • us-east4

The Insight reports refresh once a day by 21:00 UTC.

Reach out to your CSM or to [email protected] to enable or learn more about this data share feature!

CLI

❗️

These commands will overwrite a file if it exists in the path and create any missing directories or prefixes.

Users can also leverage the CLI to programmatically download all CSV reports and/or upload directly to S3.

Follow this guide to install and configure the CLI. For reference, commands, and options see here.

Supported schemas:

  • file:// - save insight locally.
  • s3:// - save insight to S3.

👍

Use get to retrieve an arbitrary insight

For CLI v0.55.0 and greater you can now use montecarlo insights get to fetch any available insight that doesn't have an explicit subcommand.

# Save an insight locally to a directory called 'mc_data' with filename 'assets.csv'
$ montecarlo insights get-key-assets file://mc_data/assets.csv

# Save an insight to S3 bucket called 'bucket' with key 'mc_data/alerts.csv'
$ montecarlo insights get-events s3://bucket/mc_data/alerts.csv

# List all insights details and availability
$ montecarlo insights list
╒══════════════════════════════════════════════════════════════════╤══════════════════════════════════════════════════════════════════════════════════════════════════════╤═════════════╕
│ Insight (Name)                                                   │ Description                                                                                          │ Available   │
╞══════════════════════════════════════════════════════════════════╪══════════════════════════════════════════════════════════════════════════════════════════════════════╪═════════════╡
│ Key Assets (key_assets)                                          │ Tables and views with a calculated Importance Score (based on # dependencies, avg. reads/writes,     │ True        │
│                                                                  │ users, and more).                                                                                    │             │
├──────────────────────────────────────────────────────────────────┼──────────────────────────────────────────────────────────────────────────────────────────────────────┼─────────────┤
│ Coverage Overview (monitors)                                     │ Active monitors that Monte Carlo is applying to your data, including all table and advanced │ True        │
│                                                                  │ monitors.                                                                                            │             │
├──────────────────────────────────────────────────────────────────┼──────────────────────────────────────────────────────────────────────────────────────────────────────┼─────────────┤
│ Table Cleanup Suggestions (cleanup_suggestions)                  │ Dormant tables and views with no recent query activity.                                              │ True        │
├──────────────────────────────────────────────────────────────────┼──────────────────────────────────────────────────────────────────────────────────────────────────────┼─────────────┤
│ Field-level Cleanup Suggestions (field_cleanup_suggestions)      │ Dormant fields with no recent query activity.                                                        │ True        │
├──────────────────────────────────────────────────────────────────┼──────────────────────────────────────────────────────────────────────────────────────────────────────┼─────────────┤
│ Field importance scores (field_importance_scores)                │ Fields with a calculated importance score based on BI usage, explicit and implicit lineage usage and │ True        │
│                                                                  │ number of read queries.                                                                              │             │
├──────────────────────────────────────────────────────────────────┼──────────────────────────────────────────────────────────────────────────────────────────────────────┼─────────────┤
│ Events (events)                                                  │ Details for all anomalies and schema changes detected by Monte Carlo, which then get grouped         │ True        │
│                                                                  │ together into alerts.                                                                             │             │
├──────────────────────────────────────────────────────────────────┼──────────────────────────────────────────────────────────────────────────────────────────────────────┼─────────────┤
│ Rule Results (rule_and_sli_results)                              │ Log of query outputs and pass/breach results for SQL, Freshness and Volume Rules for the past 90     │ True        │
│                                                                  │ days.                                                                                                │             │ 
├──────────────────────────────────────────────────────────────────┼──────────────────────────────────────────────────────────────────────────────────────────────────────┼─────────────┤
│ Field Health Suggestions (insight_monitor_recom_fh_tables)       │ Top 100 table recommendations for Field Health monitors, based on Table Importance Score (see Key    │ True        │
│                                                                  │ Assets). Note that we exclude external tables as they tend to consume more compute.                  │             │
├──────────────────────────────────────────────────────────────────┼──────────────────────────────────────────────────────────────────────────────────────────────────────┼─────────────┤
│ Misconfigured Monitors (insight_monitor_issues_and_solutions)    │ Custom monitors (e.g. Field Health) that were configured in a way that won't result in meaningful    │ True        │
│                                                                  │ anomaly detection, along with suggestions on how to fix them.                                        │             │
├──────────────────────────────────────────────────────────────────┼──────────────────────────────────────────────────────────────────────────────────────────────────────┼─────────────┤
│ BI Dashboard Analytics (bi_dashboard_analytics)                  │ Usage data for Looker dashboards such as totals views and days since last view.                      │ True        │
├──────────────────────────────────────────────────────────────────┼──────────────────────────────────────────────────────────────────────────────────────────────────────┼─────────────┤
│ Alert History (alert_history)                                    │ All incidents detected by Monte Carlo over the last 6 months.                                        │ True        │
├──────────────────────────────────────────────────────────────────┼──────────────────────────────────────────────────────────────────────────────────────────────────────┼─────────────┤
│ Notifications by Custom Monitor (custom_monitor_notifications)   │ Count of notifications each week, separated by the distinct custom monitors that generated them.     │ True        │
╘══════════════════════════════════════════════════════════════════╧══════════════════════════════════════════════════════════════════════════════════════════════════════╧═════════════╛

# Save an arbitary arbitrary insight locally to a directory called 'mc_data' with filename 'incident_history.csv
$ montecarlo insights get --name incident_history --destination file://mc_data/incident_history.csv

API or SDK

📝

Databricks and BigQuery users can use these cookbooks to help simplify importing insights!

After following this guide to create an access key users can also leverage our API or SDK to retrieve insights:

Insights Inventory

Key Assets
Tables and views with a calculated Importance Score (based on # dependencies, avg. reads/writes, users, and more). The lookback window for this insight is 30 days.
Use this to:

  • Identify important tables to add custom monitors to
  • Communicate table/view usage to stakeholders
  • Prioritise datasets for refactoring or migrations

Coverage Overview
Active monitors that Monte Carlo is applying to your data, including all Table monitors and Advanced monitors.
Use this to:

  • Quantify MC monitor coverage across schemas and tables
  • Report increase in custom monitor adoption over time

Table Cleanup Suggestions
Dormant tables and views with no recent query activity
Use this to:

  • Reduce storage costs by deleting unused tables
  • Deprioritize unused datasets and tables during migrations

Field-level Cleanup Suggestions
Dormant fields with no recent query activity.
Use this to:

  • Prioritize which tables are most ripe for cleanup
  • Simplify tables by removing unused fields

Events
Details for all anomalies and schema changes detected by Monte Carlo, which then get grouped together into alerts. The lookback window for this insight is 90 days.
Use this to:

  • Understand the threshold for what triggered an anomaly
  • See which tables or views are consistently changing or unreliable

Rule Results
Log of query outputs and pass/breach results for SQL Rules and SLIs for the past 90 days
Use this to:

  • Track progress towards SLAs or SLOs by aggregating SLI results over time
  • Fine-tune Rules by understanding Rule results over time

Field Health Suggestions
Top 100 table recommendations for Field Health monitors, based on Table Importance Score (see Key Assets). Note that we exclude external tables as they tend to consume more compute.
Use this to:

  • Identify tables for Field Health monitors

Misconfigured Monitors
Custom monitors (e.g. Field Health) that were configured in a way that won't result in meaningful anomaly detection, along with suggestions on how to fix them.
Use this to:

  • Ensure custom monitors are working as intended
  • Clean up outdated custom monitors

BI Dashboard Analytics
Usage data for Looker dashboards such as totals views and days since last view.
Use this to:

  • See which dashboards are safe to deprecate or delete
  • Prioritise heavily used dashboards in a migration

Alert History
All alerts detected by Monte Carlo over the last 6 months. The lookback window for this insight is 1 year.
Use this to:

  • Report on the statuses of your Alerts
  • Calculate Alerts Response Rate and Time to First Response
  • See trends in your anomalies and schema changes

Field importance scores
Fields with a calculated importance score based on BI usage, explicit and implicit lineage usage and number of read queries.
Use this to:

  • Identify important fields to add custom monitors to
  • Communicate field usage to stakeholders
  • Prioritize fields for refactoring or migrations

Notifications by Custom Monitor
Count of notifications each week, separated by the distinct custom monitors that generated them.
Use this to:

  • Identify monitors that may be causing alert fatigue

Updated 2 days ago