Guides
HomeChangelogStatus PageAPI ExplorerAPI ReferenceCLI ReferenceBlogRequest a Demo

Introducing: Alerts

Suggest Edits

On June 4 2024, the current state of "Incidents" in Monte Carlo will be repurposed as "Alerts." The current use of "Severity" will also split into "Priority" for Monitors and Alerts, and "Severity" will remain as the way to mark an Alert as an Incident.

Why the change?

We heard clearly from our customers that not every alert/notification from MC is an "Incident." We also know that reporting and retrospectives on "alerts" is not always necessary, but these activities on true data incidents are critical to the success of data teams.

We're better aligning Monte Carlo to accepted industry tooling and terminology: triage alerts and escalate them to incidents where it makes sense, report on those incidents, and communicate them out to stakeholders if appropriate.

What is changing?

Summary

  • "Incidents" today become "Alerts" going forward.
  • Custom Monitors can have a pre-set "Priority," replacing the pre-set "Severity" today.
  • "Priority" from a Custom Monitor is inherited to an Alert, but is not changeable on the Alert.
  • "Alerts" can be marked as "Incidents" by utilizing "Severity." This workflow will be further improved in the coming weeks.
  • The "investigating" status has been renamed to "acknowledged" in the UI, Slack, and MS Teams.

Details & Impacts of Changes

In this first release, the following may require customer action:

  • The names of insights reports are changing and may require action if you have scripted programs to extract them.
    • incident_history to alert_history: the former will continue to exist for now, but is considered deprecated and replaced with alert_history. The new insight contains columns referencing alert_* rather than incident_*. It also contains the new priority field that is inherited from the Monitor.
    • Events (events.csv): the Severity field will no longer be populated. The Priority field will be populated going forward if inheriting a monitor priority.
    • Rule Results (rule_results.csv): the alert_id field was added. The incident_id still exists but is considered deprecated.
  • Past Alerts will be backfilled with Priority values if the past Alert (Incident) had a Severity marked. No past Alerts (Incidents) will be marked with a Severity. If reporting off of Severity outside of Monte Carlo using our insights, Snowflake Data Share, or our API, you will need to leverage the Priority field going forward.
  • If you leverage the API to retrieve Severity today, we recommend starting to use Priority going forward. The existing Severity field will not be immediately deprecated.
  • If reporting off of Severity outside of Monte Carlo using our insights, Snowflake Data Share, or our API, you should to leverage the Priority field going forward.

In this first release, the following additional changes will be made, which should not require customer action:

  • Around the application, the word "Incident" has generally been fully replaced with "Alert."
  • When creating a Custom Monitor, instead of setting a Severity of SEV-0 through SEV-4, you can now set a Priority of P1 through P5.
    • For any existing Monitors as Code, Monte Carlo will automatically map these Severities to Priorities.
    • API has both "severity” and “priority” for now. Updating monitor “severity” via API updates monitor “priority” for backwards compatibility.
  • When an Alert is created, the Alert will inherit the Priority of the Custom Monitor (replacing the inheritance of Severity).
  • When an Alert is created, the Alert will not inherit any Severity. Severity now must be marked by a user, implying that this Alert constituted a true Incident.
  • Dashboards and Alerts can be filtered by Priority.
  • The URLs for /incidents will be remapped to /alerts but any existing bookmarks will automatically forward to the new URLs.
  • The "investigating" status has been renamed to "acknowledged" in the UI, Slack, and MS Teams. The

Coming soon:

  • Changes to the incident response flow: ability to mark Incidents in the UI and notification applications like Slack and MS Teams.
  • Filtering by Severity in addition to Priority around the application.

What actions must customers take?

Breaking changes:

  • If leveraging the API to extract Insights Reports, the names of the reports x and y are changing and may cause scripts to break that are extracting these insights. We are separately reaching out to users that are extracting these today to communicate this change directly.
  • If reporting off of Severity outside of Monte Carlo using our insights, Snowflake Data Share, or our API, you will need to leverage the Priority field going forward. The Severity field now is used differently.

In future:

  • For any monitors-as-code users, pre-set Severity values may need to eventually be mapped to Priority values as this may cause confusion for customers. Monte Carlo will not enforce this change in the immediate future and will map them as noted above.
  • For any external reporting on "investigating" status, use the acknowledged status going forward. This has not yet been renamed in the insights reports or API.

What is not yet changing?

In this first release:

  • The Monte Carlo API will not be affected. Customers and Partners can still use the Incidents related APIs. This will change in the coming months but we will provide an additional cutover plan well before deprecating any legacy APIs.
  • The statuses of Alerts.

Timeline

The target date for the application terminology to change is June 4, 2024. Additional changes are targeted in June and July.

Resources & Support

Please reach out through your usual support channels if you have you have any questions or concerns and we can ensure they are prioritized quickly during this change.

Updated 6 months ago