Google ADK Integration

Overview

Google's Agent Development Kit (ADK) emits OpenTelemetry traces for every agent run. Monte Carlo reads those traces from your warehouse to power Agent Observability — prompts, completions, tool calls, token usage, and conversation grouping.

Both ADK runtimes are supported:

• Python — google-adk
• Go — google.golang.org/adk ("adk-go")

This page covers the ADK-specific setup. Traces go either to Monte Carlo's hosted collector or to your own collector — see Agent Monitors Overview for the end-to-end pipeline.

The one thing to configure

Monte Carlo ingests OpenTelemetry traces and reads prompts and completions from gen_ai.* span attributes. Out of the box, neither ADK runtime puts content there — Python needs the settings below to route it onto span attributes, and Go emits content only on the OpenTelemetry logs signal, which isn't ingested. So out of the box your traces arrive with the agent's structure but without the prompts and completions.

Getting content onto span attributes is the core of ADK onboarding, and the steps differ by runtime.

Python ADK

Choose your setup

How you instrument depends on what starts the process, because OpenTelemetry's global tracer provider can be set only once and whoever sets it first wins.

GKE and other container platforms follow the first or second row, depending on whether the container runs ADK's server or your own entrypoint.

The examples below use Monte Carlo's hosted collector, https://integrations.getmontecarlo.com/otel/v1/traces, with the x-mcd-id / x-mcd-token credentials issued at integration setup, the same key id and token the Monte Carlo SDK reads from MCD_DEFAULT_API_ID / MCD_DEFAULT_API_TOKEN. If you run your own collector, use its endpoint instead, ending in /v1/traces; no Monte Carlo credentials are required (access is network-restricted).

Capture prompts and completions

Every setup needs both of these environment variables. Either one alone yields spans with token counts but no message content:

OTEL_SEMCONV_STABILITY_OPT_IN=gen_ai_latest_experimental
OTEL_INSTRUMENTATION_GENAI_CAPTURE_MESSAGE_CONTENT=SPAN_ONLY

• OTEL_SEMCONV_STABILITY_OPT_IN=gen_ai_latest_experimental opts into OpenTelemetry's GenAI semantic conventions, which are still experimental and off by default. Opting in is what makes ADK emit prompts and completions as the gen_ai.input.messages / gen_ai.output.messages / gen_ai.system_instructions attributes Monte Carlo reads.
• OTEL_INSTRUMENTATION_GENAI_CAPTURE_MESSAGE_CONTENT=SPAN_ONLY writes captured content onto span attributes (rather than log events).

Also set OTEL_SPAN_ATTRIBUTE_COUNT_LIMIT=1024. GenAI spans exceed OpenTelemetry's default limit of 128 attributes, and attributes past the limit are dropped silently. mc.setup() sets this for you.

ADK server

When adk web, adk api_server, or an adk deploy target runs your agent, ADK's server exports traces itself once an OTLP endpoint variable is set. No code changes are needed.

1. Install the OTLP exporter. ADK imports it for this path, but since ADK 2.0 google-adk doesn't depend on it (it moved to the gcp and all extras). Installing montecarlo-opentelemetry also brings in the exporter.

   Bash

   pip install "google-adk[gcp]"        # or: pip install opentelemetry-exporter-otlp-proto-http

   adk deploy cloud_run and adk deploy gke install only google-adk in the image, so add opentelemetry-exporter-otlp-proto-http to the agent folder's requirements.txt. Without it, the server exits at startup with ModuleNotFoundError: No module named 'opentelemetry.exporter'.

2. Set these in the environment that launches the server, plus the capture variables above:

   Bash

   OTEL_EXPORTER_OTLP_TRACES_ENDPOINT=https://integrations.getmontecarlo.com/otel/v1/traces
   OTEL_EXPORTER_OTLP_TRACES_HEADERS=x-mcd-id=<key id>,x-mcd-token=<token>
   OTEL_SERVICE_NAME=<your agent's name>
   OTEL_SPAN_ATTRIBUTE_COUNT_LIMIT=1024

   Use OTEL_EXPORTER_OTLP_TRACES_ENDPOINT, not OTEL_EXPORTER_OTLP_ENDPOINT. The generic variable also turns on ADK's metrics and logs exporters, which Monte Carlo rejects, so your logs fill with Failed to export ... code: 403 errors. The traces variable is used exactly as given, so it must end in /v1/traces. Likewise, use OTEL_EXPORTER_OTLP_TRACES_HEADERS rather than OTEL_EXPORTER_OTLP_HEADERS so the credentials go only to trace exporters; the generic variable is sent by every OTLP exporter in the process that doesn't set its own headers.

   ADK reads these variables when the server starts. An agent's .env file is loaded later, on the first request, so variables set only there have no effect: traces aren't exported, or the agent appears as unknown_service or with no name.

Prefer the Monte Carlo SDK? Instead of the endpoint variables, add the Monte Carlo OpenTelemetry SDK to the agent module. ADK has already created the tracer provider by the time it imports your agent, so attach to that provider rather than creating one:

import montecarlo_opentelemetry as mc

mc.add_processor()
mc.add_exporter("https://integrations.getmontecarlo.com/otel/v1/traces")  # reads MCD_DEFAULT_API_ID / MCD_DEFAULT_API_TOKEN

This adds the processor that copies montecarlo.workflow / montecarlo.task baggage onto spans, plus the SDK's configuration warnings. Don't also set an OTEL_EXPORTER_OTLP_*ENDPOINT variable, or every span is sent twice.

On Cloud Run

When deploying with adk deploy cloud_run:

• Keep credentials in Secret Manager. With the environment-variable setup, the whole OTEL_EXPORTER_OTLP_TRACES_HEADERS value is the secret. Pass it after -- as --set-secrets OTEL_EXPORTER_OTLP_TRACES_HEADERS=<secret>:latest.
• Keep .env out of the image. adk deploy copies the agent folder into the image and honors only ignore files inside that folder, so add a .gcloudignore there that lists .env.
• Override the model location if needed. adk deploy sets GOOGLE_CLOUD_LOCATION to the deploy region, so models served only from global return 404 unless you pass --env GOOGLE_CLOUD_LOCATION=global.
• Removing a variable needs --remove-env-vars. adk deploy adds variables with --update-env-vars, so switching an existing service from OTEL_EXPORTER_OTLP_ENDPOINT to the traces variable leaves the old one set.

Your own entrypoint

If your code starts the process (a custom FastAPI app or an embedded Runner), call mc.setup() before ADK builds its app, so Monte Carlo's tracer provider becomes the global one. ADK instruments itself, so you don't pass any instrumentors:

import montecarlo_opentelemetry as mc

mc.setup(
    agent_name="<your agent's name>",
    otlp_endpoint="https://integrations.getmontecarlo.com/otel/v1/traces",
)

from google.adk.cli.fast_api import get_fast_api_app  # after mc.setup()

app = get_fast_api_app(agents_dir="<folder containing your agent>", web=False)

Order matters: if get_fast_api_app() runs first, ADK's provider wins and mc.setup() logs that Monte Carlo's tracer provider was not installed. The same applies to an embedded Runner: call mc.setup() before you create it. mc.setup() reads the credentials (MCD_DEFAULT_API_ID, MCD_DEFAULT_API_TOKEN) when it runs, so they must be in the launch environment.

Agent Runtime

On Agent Runtime, Google's AdkApp.set_up() configures OpenTelemetry in each worker, wiring in Google's exporter so traces go to Cloud Trace. To send them to Monte Carlo as well, add Monte Carlo's processor and exporter to the provider Google created.

The catch is timing. set_up() runs after your code is imported and replaces the whole span pipeline, so anything attached at import time is wiped with no error: traces reach Cloud Trace but never Monte Carlo. Attach Monte Carlo after set_up() runs by subclassing AdkApp:

# mc_instrumented_app.py
import os

import montecarlo_opentelemetry as mc
from opentelemetry import trace
from opentelemetry.sdk.trace import TracerProvider
from vertexai.agent_engines import AdkApp


class McAdkApp(AdkApp):
    def set_up(self):
        super().set_up()  # Google replaces the span pipeline here, so attach Monte Carlo after it
        endpoint = "https://integrations.getmontecarlo.com/otel/v1/traces"
        if isinstance(trace.get_tracer_provider(), TracerProvider):
            mc.add_processor()  # Google's telemetry on
            mc.add_exporter(endpoint)
        else:  # telemetry turned off: no provider exists
            mc.setup(agent_name=os.environ["OTEL_SERVICE_NAME"], otlp_endpoint=endpoint)

# deploy.py
import vertexai
from google.cloud.aiplatform_v1.types import SecretRef
from vertexai import agent_engines

from mc_instrumented_app import McAdkApp
from my_agent.agent import root_agent

vertexai.init(
    project="<project>",
    location="<region>",
    staging_bucket="gs://<bucket>",  # create() uploads the app to this bucket
)

agent_engines.create(
    McAdkApp(agent=root_agent),
    requirements=["google-adk", "google-cloud-aiplatform[adk,agent-engines]", "montecarlo-opentelemetry"],
    extra_packages=["mc_instrumented_app.py", "my_agent/__init__.py", "my_agent/agent.py"],
    env_vars={
        "GOOGLE_CLOUD_AGENT_ENGINE_ENABLE_TELEMETRY": "true",
        "OTEL_SERVICE_NAME": "<your agent's name>",
        "OTEL_SPAN_ATTRIBUTE_COUNT_LIMIT": "1024",
        "OTEL_SEMCONV_STABILITY_OPT_IN": "gen_ai_latest_experimental",
        "OTEL_INSTRUMENTATION_GENAI_CAPTURE_MESSAGE_CONTENT": "SPAN_ONLY",
        "MCD_DEFAULT_API_ID": "<key id>",
        "MCD_DEFAULT_API_TOKEN": SecretRef(secret="<token secret>", version="latest"),
    },
)

• Pass configuration through env_vars=. Agent Runtime doesn't read .env files.
• Turn on Google's telemetry. GOOGLE_CLOUD_AGENT_ENGINE_ENABLE_TELEMETRY=true makes set_up() build the tracer provider that McAdkApp attaches to, and makes Agent Runtime flush it after every query. It also sends traces, including prompts and responses, to Cloud Trace, and logs to Cloud Logging.
• Set OTEL_SERVICE_NAME. Without it, the agent appears in Monte Carlo under its numeric engine ID.
• Give the subclass's module a distinctive name, not app.py. Shipped modules land in the worker's working directory, and a module named app.py makes the worker exit at startup with no error logged.
• Grant secret access. The Agent Runtime service agent (service-<project number>@gcp-sa-aiplatform-re.iam.gserviceaccount.com) needs roles/secretmanager.secretAccessor on the token secret. This service agent exists only after the project's first Agent Runtime deployment, so the first deploy may fail with PERMISSION_DENIED until you grant it.

When Google's telemetry is on, Agent Runtime flushes the tracer provider after every query, so the last spans of a conversation aren't lost. With telemetry off, McAdkApp falls back to mc.setup() and nothing flushes after a query: spans are exported on the batch timer, so the last spans can be lost if the instance is throttled.

Troubleshooting

Go ADK (adk-go)

adk-go does not emit message content on spans under any configuration — its content path is Go-side logs, which Monte Carlo does not ingest. Instead, set the content and grouping attributes directly on the active span at your shared model boundary (for example, your model.LLM implementation or AI gateway), where the span is still recording:

import (
    "go.opentelemetry.io/otel/attribute"
    "go.opentelemetry.io/otel/trace"
)

// ctx is the context passed into your model.LLM call.
span := trace.SpanFromContext(ctx)
if span.IsRecording() {
    span.SetAttributes(
        attribute.String("gen_ai.input.messages", inMsgsJSON),
        attribute.String("gen_ai.output.messages", outMsgsJSON),
        attribute.String("gen_ai.system_instructions", sysJSON),
        attribute.String("gen_ai.conversation.id", sessionID),
    )
}

• gen_ai.input.messages and gen_ai.output.messages are each a JSON string holding an array of messages. Monte Carlo accepts two shapes: a flat [{"role": "...", "content": "..."}] array, or the standard semantic-convention [{"role": "...", "parts": [{"type": "...", "content": "..."}]}] (content nested under parts). The flat shape is enough; the parts shape also works for text and tool-call parts.
• A tool call the model makes is a part {"type": "tool_call", "id": "...", "name": "...", "arguments": {...}} — arguments is a nested JSON object (a JSON-encoded string works too), and Monte Carlo reads either form.
• A tool result carried into the next turn should put the result text in the message's content (a flat {"role": "tool", "content": "..."}) — a tool_call_response part carries its payload elsewhere and won't render in the message view.
• gen_ai.system_instructions is a bare array of {"type": "text", "content": "..."} objects (no role wrapper).
• One edit at a shared model/gateway boundary covers every ADK agent behind it.
• The model name shown in Monte Carlo comes from what your model client reports — adk-go records it as gen_ai.request.model. If that is blank (common behind an OpenAI-compatible gateway, where the client's name is empty), set gen_ai.request.model yourself in the SetAttributes call above so runs are labeled with the model. Token counts likewise appear only when your client reports usage.
• Tool calls need nothing extra — adk-go already emits tool input and output natively.
• Set these attributes before your model client yields its first non-partial response — adk-go ends the generate_content span at that point, and attributes set after a span ends are dropped.

Wiring the exporter (Go)

adk-go and any HTTP instrumentation share the global TracerProvider, so a single OTLP/HTTP exporter carries every span. Where you build your provider today:

import (
    "go.opentelemetry.io/otel"
    "go.opentelemetry.io/otel/exporters/otlp/otlptrace/otlptracehttp"
    sdktrace "go.opentelemetry.io/otel/sdk/trace"
)

// ctx is your startup context.
mcExp, err := otlptracehttp.New(ctx,
    otlptracehttp.WithEndpointURL("https://<your-endpoint>/v1/traces"),
    otlptracehttp.WithCompression(otlptracehttp.GzipCompression),
    // Set the credentials as headers on THIS exporter — not via
    // OTEL_EXPORTER_OTLP_HEADERS, which applies to every OTLP exporter in
    // the process and would attach them to your other export paths.
    otlptracehttp.WithHeaders(map[string]string{
        "x-mcd-id":    mcdID,
        "x-mcd-token": mcdToken,
    }),
)
if err != nil {
    // handle the error
}

// The SDK default span attribute-count limit is 128 and overflow is dropped
// silently; raise it at provider construction.
limits := sdktrace.NewSpanLimits()
limits.AttributeCountLimit = 1024

tp := sdktrace.NewTracerProvider(
    // ... your existing options: a resource that sets service.name — this is
    // the agent's identity in Monte Carlo — plus any existing span processors ...
    sdktrace.WithBatcher(mcExp, sdktrace.WithMaxExportBatchSize(128)),
    sdktrace.WithRawSpanLimits(limits),
)
otel.SetTracerProvider(tp) // adk-go resolves the global provider

• Endpoint — OTLP over HTTP/protobuf (no gRPC). Only traces are ingested; logs and metrics — including adk-go's gen_ai.* log records — are not.
• Check the endpoint URL — if WithEndpointURL can't parse the URL it keeps the default localhost:4318 instead of returning an error, and the parse failure is only logged through OpenTelemetry's internal error logging. A bad endpoint can therefore silently export your traces to the wrong destination (or drop them) rather than failing your setup code.
• Credentials — for Monte Carlo's hosted collector (https://integrations.getmontecarlo.com/otel/v1/traces — note the /otel path), set the x-mcd-id / x-mcd-token headers on this exporter (above). A collector on your own network needs none — see Agent Observability Data Platform.
• service.name — set it on the resource; it is the agent's identity in Monte Carlo.
• Attribute limit — raise it to 1024 as shown (or OTEL_SPAN_ATTRIBUTE_COUNT_LIMIT=1024, ignored if your code already sets span limits). The default of 128 drops attributes with no error.
• Batching — export requests are rejected above ~4 MB of uncompressed span data; the 128-span batch (vs the OTel default of 512) keeps typical batches under it. Enable gzip.
• Already initializing ADK telemetry with telemetry.New() / SetGlobalOtelProviders()? adk-go emits its spans through the global TracerProvider, so register exactly one — either build your own with the Monte Carlo exporter (above), or keep ADK's helper and attach the exporter through its options. Registering both is unreliable: adk-go binds to whichever provider is set first, so a later otel.SetTracerProvider(...) may never capture adk-go's own spans (it's first-wins, not last-wins).

Grouping runs into conversations

Runs are grouped into a single conversation by gen_ai.conversation.id, and the conversation view threads together every turn that shares one:

• Python — set automatically from your ADK session id, so each session becomes one conversation. No extra work is required.
• Go — adk-go sets it on the invoke_agent span (ADK's top-level span) from the ADK session id, but not on the content spans, and it only threads turns together if that id is stable across them. Set your own stable conversation id (for example your session store's id) on the content spans in the SetAttributes call shown above, so every turn groups.

Optionally set montecarlo.workflow (the flow handling the request) and montecarlo.task (the step within it) on your spans to group and filter runs by workflow and step.

Monte Carlo reads these grouping attributes per span, with no inheritance from parent to child — so set gen_ai.conversation.id (and any montecarlo.workflow / montecarlo.task) on every span you want grouped or filtered: the root, LLM, and tool spans alike, not just one.

In Python, the Monte Carlo SDK does this for montecarlo.workflow / montecarlo.task. Wrap the work in mc.create_span_with_workflow() / mc.create_span_with_task() (or the mc.trace_with_workflow / mc.trace_with_task decorators), and the SDK's processor, added by mc.add_processor() or mc.setup(), copies the value onto every span started inside it, including ADK's. The environment-variable-only ADK server setup doesn't include that processor.

See Trace and Conversation Structure for how traces and conversations are assembled.

What gets captured

Once content is on spans and traces are flowing to your warehouse:

Next steps

1. Confirm traces are landing in your warehouse and select your trace table on the Agent Observability settings page.
2. Create Agent Evaluation, Agent Metric, Agent Trajectory, and Agent Validation monitors over your ADK agents.