Databricks Agent Bricks

Monte Carlo's Agent Observability provides full visibility into your Databricks Agent Bricks and Mosaic AI Agent Framework agents. Once you enable trace storage, Databricks logs detailed MLflow trace data to Unity Catalog, and Monte Carlo reads this data directly through your existing Databricks SQL Warehouse connection — giving you trace-level visibility, out-of-the-box dashboards, and full agent monitor coverage without deploying additional infrastructure.

What is Databricks Agent Bricks?

Databricks Agent Bricks is Databricks' managed platform for building and deploying AI agents — from ready-made templates like Knowledge Assistant to custom code-based agents. Teams can create agents that answer questions, search enterprise data, and orchestrate tools — all within their Databricks workspace. Agent runs are traced with MLflow, capturing conversation history, tool execution, LLM planning, and response generation.

Monte Carlo supports both kinds of Databricks agents, and labels them accordingly in its agent list:

• Knowledge Assistant (KA) — a managed Agent Bricks template.
• Custom agent (MLflow SDK) — your own Mosaic AI Agent Framework agents instrumented via MLflow (autolog or manual tracing).

Getting traces into Unity Catalog

📘

Prerequisites

An existing Databricks integration in Monte Carlo with a SQL Warehouse connection. If you haven't set this up yet, see our Databricks SQL Warehouse guide.

Monte Carlo reads your agents' MLflow traces from a trace table in Unity Catalog — <catalog>.<schema>.<table_prefix> for Knowledge Assistant templates, or <catalog>.<schema>.<table_prefix>_trace_unified for custom agents. Databricks logs traces automatically, but they reach Unity Catalog only once you enable trace storage. How to turn it on depends on your agent:

• Knowledge Assistant templates — enable trace syncing (Delta sync) on the template's MLflow experiment.
• Custom agents deployed via agents.deploy() — enable Production Monitoring on the agent's MLflow experiment.
• Custom code-based agents — set the experiment's trace location in your agent code (requires MLflow 3.11+):

import os
import mlflow
from mlflow.entities.trace_location import UnityCatalog

os.environ["MLFLOW_TRACING_SQL_WAREHOUSE_ID"] = "<sql-warehouse-id>"  # SQL warehouse used for trace storage
mlflow.set_experiment(
    experiment_name="<your-experiment-name>",
    trace_location=UnityCatalog(
        catalog_name="<catalog>",
        schema_name="<schema>",
        table_prefix="<table_prefix>",
    ),
)

For full setup details, see Databricks' guides for Production Monitoring and MLflow Traces in Unity Catalog.

Until trace storage is enabled, an agent's traces stay in its MLflow experiment and never reach Unity Catalog, so Monte Carlo can't read them. Monte Carlo will still discover and list the agent, but flags it as having no trace table. Once you enable trace storage, traces from new agent runs land in the Unity Catalog table and become visible in Monte Carlo within about 30 minutes.

Permissions

Monte Carlo authenticates to Databricks with the principal already attached to your SQL Warehouse connection — no extra credentials. That principal needs:

• To read traces: read access to the trace table — its catalog, schema, and the table itself (the bare <table_prefix> for Knowledge Assistant templates, or <table_prefix>_trace_unified for custom agents). If the principal already has read access to the catalog and schema where traces land (the common case), nothing extra is needed.
• To discover agents in the picker: the connection principal only lists agents it can see in Databricks — by default, just the ones it created or was granted access to. For each agent you want to monitor, grant the principal CAN VIEW on its serving endpoint (Knowledge Assistant) or CAN READ on its MLflow experiment (custom agent), from that object's Permissions dialog in Databricks. A dedicated, least-privilege service principal — granted exactly these per-agent permissions rather than broad workspace access — is the recommended connection identity.

Adding the agent in Monte Carlo

1. In Monte Carlo, navigate to Settings → Agent Observability
2. Click Add
3. Toggle the Agent Type to Platform Agent
4. Select a Databricks SQL Warehouse: Choose the warehouse where your trace table lives
5. Select an Agent: Select your agent from the discovered list (if it's not listed, see Permissions above)
6. Click Import to complete the connection

Once connected, Monte Carlo begins ingesting trace and span data and you're ready to create Agent Monitors.

Segmenting traces with workflow and task tags

Custom agents can optionally tag their MLflow spans with the montecarlo.workflow and montecarlo.task attributes to unlock facet filtering, segmentation, and monitor scoping in Monte Carlo. workflow names the high-level business flow; task names a sub-step within it. The attributes apply only to the span they're set on — child spans (including autolog-generated LLM spans) don't inherit them — so set them on the spans you want to filter, segment, or monitor.

import mlflow

# Option A — set on the current span
with mlflow.start_span(name="retrieve_context") as span:
    span.set_attribute("montecarlo.workflow", "support-bot")
    span.set_attribute("montecarlo.task", "retrieve-context")
    ...

# Option B — set on a traced function via the decorator
@mlflow.trace(attributes={
    "montecarlo.workflow": "support-bot",
    "montecarlo.task": "final-answer",
})
def answer(question): ...

Tagging is fully optional — untagged spans simply leave Workflow and Task empty in Monte Carlo.

Grouping turns into conversations

Custom agents can group multi-turn interactions into a single conversation in Monte Carlo by stamping each trace with a session ID — your application's conversation identifier (for example, a chat thread ID), reused across every turn:

import mlflow

mlflow.update_current_trace(metadata={"mlflow.trace.session": "<your-conversation-id>"})

Without it, each turn appears as a standalone conversation.