Troubleshooting & FAQ: Databricks

For setup and general questions, see Databricks.

I received a "Privilege SELECT is not applicable" error granting permissions in Unity Catalog. What should I do?

If the command results in an error like the one below, your UC metastore was created during the UC public preview period.

Privilege SELECT is not applicable to this entity [...:CATALOG/CATALOG_STANDARD]. If this seems unexpected, please check the privilege version of the metastore in use [0.1].

Follow Databricks instructions to upgrade the privilege model to 1.0 or configure permissions according to the old model.

Why did queries stop appearing in Monte Carlo?

Databricks masks query text in system.query.history for principals outside the databricks_pii_access group. If recent queries no longer appear on a table's Queries tab, check that the Monte Carlo service principal is a member of that group. See Query text access. If you have more than one Databricks integration, check the service principal of each.

Why do I see different lineages between Unity Catalog and Monte Carlo?
Monte Carlo builds lineage using the system tables provided from Unity Catalog. We expire nodes that have not seen an update for 7 days to de-noise the lineage, whereas Databricks does not expire the nodes for 1 year. This might cause an infrequently updated node to not show up in MC lineage but remain in Unity lineage. Please feel free to reach out to us if this is an issue for your use case.

Why do I see duplicate Databricks jobs?

This usually happens when you have multiple Databricks integrations connected to different workspaces, but the Workspace ID is not specified in one or more of the connections.

To fix this:

1. Go to Settings → Integrations.
2. Click Edit on your Databricks integration.
3. Enter the correct Workspace ID.
4. Click Save.

This will ensure each integration is linked to the right workspace and prevent job duplication.

How long do Databricks jobs and workflows stay visible on the Assets page if they don’t have new runs?

Jobs and workflows will remain listed on the Assets page as long as they have at least one run in the past seven days. If a job has had no runs for more than seven days, it will still be stored in our database—but it won’t appear on the Assets page until a new run occurs. Once you trigger a new run, the job and all of its past runs will automatically reappear.

Why am I seeing freshness updates if I only ran an OPTIMIZE or a VACUUM query, which doesn’t change the data in my table?

Monte Carlo retrieves freshness information using DESCRIBE DETAIL queries. Databricks updates the last altered time of a table when OPTIMIZE or VACUUM queries run. As a result, Monte Carlo interprets these changes as freshness events, even though the table’s content hasn’t actually changed.

Why do my foreign catalogs show no tables?

Monte Carlo needs column-level access to collect foreign (Lakehouse Federation) tables. See Foreign catalogs to grant access or exclude the catalog.