Locally setting up Monitors as Code
This tutorial will go over the steps to locally set up Monitors as Code. Monitors as Code is a great way codify monitors at scale without needing to go into the Monte Carlo UI.
Last Updated: April 11, 2022
In this example, I'll be leveraging dbt, and I'll also be making the assumption that you've downloaded our CLI. If you have not, please see here: Using the CLI
The first step here will be to create a
montecarlo.yaml file, in which you'll want to pass through
1. For most of you that will be sufficient; if you do have multiple data warehouses set up within Monte Carlo, you will also want to add this
default_resource in which you can explicitly define which data warehouse you are using for these monitors. Secondarily, you'll want to set up a yaml file for these monitors. In this specific case, I've created the
monitors.yaml file in which I'm defining both a Field Health as well as a Dimension Tracking monitor:
Opt In Monitors
Read more about our opt in monitors, including Field Health and Dimension Tracking here: Monitors Overview
Alternatively, you can also embed these monitors within your dbt
schema.yaml file. For more details on what that looks like as opposed to this example, you can go to the "Defining Individual Monitors" section within our Monitors as Code page within our documentation.
Once you have that set up, you'll first want to navigate to the location where you have your
montecarlo.yaml file, as that is the project root; in this case, we can see that I have that here. And now to set these up, I will want to run the
montecarlo monitors apply command. In this case, I am going to define this under a test namespace here so that we can see this within the UI. Now, one thing to note here is with these namespaces, this is a way of naming a grouping of monitors you are creating so that the system can synchronize changes between your local configuration and what's stored within the Monte Carlo database:
As I run this, we can see that it catches the Field Health monitor as well as the Dimension Tracking monitor and these changes were successfully applied:
Now if I go to the Monte Carlo environment and refresh the page, we'll be able to see those two monitors at the top of the list of customer monitors here. As you can see, the namespace matches, as well as the table, and the type of monitor:
Now if i also wanted to remove these, I can do so leveraging the
montecarlo monitors delete command, and I'll want to match the namespace so it knows what it should delete:
Now if I refresh my Monte Carlo UI, you'll be able to see that those no longer exist within the Monte Carlo environment:
Hopefully this was helpful, but please feel free to reach out to [email protected] or the chat bot in the lower right hand corner if you have any more questions!
Please be aware these monitors will be applied to any alert routing already in place. For example, if you have routes that get alerts on a schema or database level, setting up 500 monitors at once may cause an influx in emails, slack pings, team messages, etc.
Question: I am getting an error when trying to set up hundreds of monitors, how can I fix this?
Answer: Currently, we are limited in the number of monitors that can be set up concurrently. The workaround is to set up multiple different namespaces of ~50 monitors or less. This will also reduce singularity around your monitors, allowing you to edit a few monitors at a time as opposed to editing all.
Updated about 2 months ago