Databricks SQL Warehouse

Introduced in: CLI v0.39.2

📘

Prerequisites

To complete this guide, you will need admin rights in your Databricks workspace.

Monte Carlo can leverage a Databricks SQL Warehouse for running monitors (e.g. custom SQL, field health, etc.)

Create Databricks SQL Warehouse

  1. Follow these steps to create a SQL Warehouse. The sizing of that warehouse depends on the type and number of monitors that you wish to use within Monte Carlo.
  2. Save the Warehouse ID.
1750

Create a Databricks SQL Warehouse wizard example

Create an API key

  1. Follow these steps to generate an API key in your workspace with no specified lifetime. Monte Carlo recommends you create a service account associated with the token first.
  2. Save the generated token.
975

Create a token wizard example

Add a Databricks SQL Warehouse connection to MC

Provide connection details for the Databricks SQL Warehouse using Monte Carlo's CLI:

  1. Follow this guide to install and configure the CLI. Requires >= 0.39.2.
  2. Use the command add-databricks-sql-warehouse to set up Databricks SQL Warehouse connectivity. For reference and command options see the docs here. Below is an example:
montecarlo integrations add-databricks-sql-warehouse --databricks-workspace-url dbc-12345678-abcd.cloud.databricks.com --databricks-warehouse-id 1234567890abcdef --databricks-token -1 --name data-lake