To complete this guide, you will need admin rights in your Databricks workspace.
Monte Carlo can leverage a Databricks SQL Warehouse for running monitors (e.g. custom SQL, field health, etc.)
- Follow these steps to create a SQL Warehouse. The sizing of that warehouse depends on the type and number of monitors that you wish to use within Monte Carlo.
- Save the Warehouse ID.
- Follow these steps to generate an API key in your workspace with no specified lifetime. Monte Carlo recommends you create a service account associated with the token first.
- Save the generated token.
If you're setting up an entirely new Databricks integration, including a SQL warehouse is a recommended part of the main integration workflow.
To add a SQL Warehouse to an existing Databricks integration, start creating a new Databricks connection in the Settings-Integration page, and then select "Add to existing integration"
Provide connection details for the Databricks SQL Warehouse using Monte Carlo's CLI:
- Follow this guide to install and configure the CLI. Requires >=
- Use the command
add-databricks-sql-warehouseto set up Databricks SQL Warehouse connectivity. For reference and command options see the docs here. Below is an example:
montecarlo integrations add-databricks-sql-warehouse --databricks-workspace-url dbc-12345678-abcd.cloud.databricks.com --databricks-warehouse-id 1234567890abcdef --databricks-token -1 --name data-lake
Updated about 1 month ago