Three key improvements were just shipped for the Monte Carlo <> Jira integration:

  • Support for required fields. Previously, the integration did not natively support required fields, so if your Jira environment required more than the minimum to create an Issue, you could not proceed. Now, any required fields are surfaced directly in Monte Carlo so the user can fill them out.
  • “Owner” of the Jira issue can now be assigned from within Monte Carlo. Previously, owner was defaulted to the user that had set up the API key used for the integration.
  • More alert information is included in the description. The information we pass into the description is now at parity with what Monte Carlo would send in a notification, making is easier to understand what has happened.

We've made it easier to select what Audiences a Data Product's out-of-the-box detectors should route to, right from the creation and edit flow.

Now, when you create a Data Product, you will be able to select an existing Audience (or create a new one) to automatically route alerts to from the Data Product. This is essentially handling the creation of a Notification Rule under "Other notifications" to route "Affected data" by that Asset tag for the Data Product to that Audience.

Additionally, on the Data Product Coverage Tab you can see which Audiences are associated with this Data Product.
Note: We only show the Audiences where there is a Notification rule that specifies only that Data Product Asset tag and no other conditions. i.e. if the Notification Rule said "Include Data Product tag but exclude this other tag" it would not show up in this section.

Custom monitor failure notifications due to misconfigurations used to be limited to a frequency of 1 notification per week. We've changed the frequency to up to 1 notification per day in order to reduce the risk of monitor failing silently while still being mindful of alert fatigue.

We’ve improved our automated freshness monitor to better handle weekly patterns. We've sometimes heard customers refer to these as “bimodal” thresholds… meaning one threshold during weekdays, and a different threshold during weekends. These are valuable when the table updates during the week, but not (or less frequently) on the weekend.

Specific changes:

  • We more accurately recognize weekends. We previously had hardcoded support for Sat/Sun weekends. Now, we dynamically recognize weekends regardless of where they fall in the week (e.g. Fri/Sat, or when there’s just a single day “off”)
  • Tighter thresholds in the “active” period of the week. About 5% of tables that we’re tracking have had their thresholds reduced by >25% from this change!
  • Thoughtful thresholds in the “off” part of the week. So if the table doesn’t start back up on Monday, we’ll alert.

In the future, we’ll extend these improvements to “Time since last row count change”, and also add bimodality for day/night behaviors.

MC now supports Databricks oAuth M2M authentication with setup available via the CLI (UI to come).
Follow docs here to authenticate your MC Service Principal via M2M.

Next week, the UI of the login page will be refreshed. All functionality, links, and workflows from the login page remain the same. The changes are cosmetic only.

New login page

New login page

We are about to remove the two widgets at the top of the monitors page as the information they present appears in other dashboards in the product. This is the first step in our overhaul of the monitor list to improve usability and simplify monitor management workflows.

MC can now manage failures from Databricks Workflows. This helps users triage and resolves all data and system issues in one single platform. Follow docs here to set up the webhook connection to start alerting.