Conversation Clusters

A cluster is a group of related conversations. Clustering works at the conversation level, across the whole multi-turn exchange rather than any single trace within it, and clusters are built for you as new conversations come in. Instead of reading conversations one at a time you can see what people are asking this agent, and where it's falling down, across all of them at once.

Clusters are available under Assets → Agents → select an agent → Conversations. The Clusters panel sits above the conversation list, with two views:

• Intent — what users asked for, discovered from your agent's own traffic.
• Issues — how it went. Failure modes based on how conversations actually performed.

Select a cluster and the list below filters to it, so a pattern on a card is one click from the conversations behind it.

Clusters are useful when you want to:

• Discover what your agent is really used for. The intents that come back are rarely the ones the agent was scoped to handle, and the gap is the interesting part.
• Read the conversations most likely to be broken. The Issues view ranks the slow, expensive, and looping ones so you're not sampling at random.
• Watch a specific kind of request. Define a custom cluster for a new feature, a compliance-sensitive ask, or an escalation path, and track its share over time.
• Notice when usage shifts. Cluster shares move as your users do, so you can watch a kind of request grow or fade over time.

Intent: what users are asking

Clustering analyzes conversations, works out what your end users are asking about, names each intent it finds, and sorts every new conversation into one as it arrives. No tagging, no instrumentation, no taxonomy to maintain.

Each cluster includes the intent, its share of conversations, a description of what it covers, and how many conversations are in it. Click a card to filter the list to those conversations. The conversation table also has an Intent cluster column, so you can see where each conversation landed without filtering at all.

Custom intent clusters: track what matters to you

Custom clusters let you define clusters of your own, so anything you specifically want to watch gets its own grouping: a feature you just launched, a compliance-sensitive request type, an escalation path. They sit alongside the discovered ones and are classified the same way.

Creating, editing, and deleting them requires monitor write permission.

A custom cluster has a name and a description. The description is key: it's what conversations get matched against. Be specific about the requests you want caught, the way you'd describe them to a colleague. A description like Billing questions will pull in anything money-adjacent; one like this won't:

Users asking to close, freeze, or permanently delete their account, including requests routed through a support escalation.

You can create up to 10 custom clusters per agent, and they appear at the top of the list with a Custom badge.

Editing a cluster changes what happens next, not what already happened. Reword a description and future conversations get matched against the new wording; conversations already sorted stay where they are. To re-sort history under a new definition, delete the cluster and create it again.

Issues: where the agent is failing

Issues clusters show which conversations went poorly, whatever the user was asking for. These clusters are pre-defined:

A conversation can land in more than Issue cluster: slow and token-heavy and stuck in a tool loop. In the conversations list, the Issues column includes every issue type a conversation hit, and the ones carrying three at once are usually where to start.

Long-running and Token-heavy stay populated by design: they always hold this agent's current worst, so the queue stays useful as the agent changes.

Low evaluation score clusters work the other way — they hold whatever your Agent Evaluation Monitors scored below the bar, scoped to evaluate entire conversations, one cluster per dimension. A monitor checking helpfulness and completeness gives you two clusters. They appear once that evaluation starts producing scores, and an empty one is good news.

Cluster performance

Every cluster carries the aggregated performance of the conversations inside it, so you can tell which of your agent's jobs is going badly before opening any of them: execution duration, elapsed duration, tokens, turns, and error rate — at median and p95, each with a delta against all conversations (▲ 260% vs all).

A metric is highlighted when it sits well off the baseline for all conversations on this agent. Which number moved tells you what to read for:

Available on both Intent and Issues clusters.

Monitoring a cluster

Agent Metric Monitors and Agent Evaluation Monitors can both scope to a conversation cluster (intent or issue) — so a monitor watches one kind of request instead of the agent's whole traffic.

• Filter by cluster narrows a monitor to one or more clusters. Use it when a group of conversations should be held to a different bar than the rest: an eval judged on what good refund handling looks like, rather than one judge scoring refunds and return-policy questions the same way. It also keeps evaluation cost down, since the judge only runs over the conversations you picked.
• Segment by cluster creates one monitor with a series per cluster, so you can compare them and catch shifts you weren't watching for — a regression in one intent that a healthy average would hide, or a cluster going from 3% of traffic to 50%. New clusters arrive as new series automatically.

Both options are in the scope step when creating or editing an agent metric or evaluation monitor.

FAQ

The Clusters panel is empty. Is clustering not enabled?

Not necessarily — check whether you've selected a workflow first. An agent running several workflows is clustered per workflow. Use the workflow filter on the Conversations tab and that workflow's clusters appear. Each workflow has its own topic set.

If you've selected a workflow and see an Enable clustering button, clustering isn't on for that agent yet. Go ahead and enable it.

Can I get alerted when a cluster starts behaving badly?

Yes. Scope an agent metric or evaluation monitor to the cluster — filter to it to watch it on its own, or segment by cluster to get a series per cluster from one monitor. See Monitoring a cluster.

Long-running always sits at about 5%. Is that bad?

No — that's the resting state, because the threshold is relative to this agent's own recent traffic.

I edited a custom cluster's description. What happens to a monitor scoped to it?

The monitor keeps running, and its series continues uninterrupted. New conversations are matched against the new wording; conversations already sorted stay where they are.

For tightening wording, that's exactly what you want. If you're materially changing what the cluster catches — from billing questions to account closures, say — create a new cluster instead, so the monitor's history reflects a single definition throughout.

Related

• Trace and Conversation Structure — how traces become conversations
• Agent Evaluation Monitors — quality scoring, which feeds Low evaluation score
• Agent Metric Monitors — alerting on latency, tokens, and errors
• Agent Monitors Overview — setup and monitor types