Conversation Clusters

A cluster is a group of related conversations. Clustering works at the conversation level, across the whole multi-turn exchange rather than any single trace within it, and clusters are built for you as new conversations come in. Instead of reading conversations one at a time you can see what people are asking this agent, and where it's falling down, across all of them at once.

Clusters are available under Assets โ†’ Agents โ†’ select an agent โ†’ Conversations. The Clusters panel sits above the conversation list, with two views:

  • Intent โ€” what users asked for, discovered from your agent's own traffic.
  • Issues โ€” how it went. Failure modes based on how conversations actually performed.

Select a cluster and the list below filters to it, so a pattern on a card is one click from the conversations behind it.

Clusters are useful when you want to:

  • Discover what your agent is really used for. The intents that come back are rarely the ones the agent was scoped to handle, and the gap is the interesting part.
  • Read the conversations most likely to be broken. The Issues view ranks the slow, expensive, and looping ones so you're not sampling at random.
  • Watch a specific kind of request. Define a custom cluster for a new feature, a compliance-sensitive ask, or an escalation path, and track its share over time.
  • Notice when usage shifts. Cluster shares move as your users do, so you can watch a kind of request grow or fade over time.
๐Ÿ“˜

Before you start

Because clustering works at the conversation level, your agent needs conversation grouping configured first โ€” see Trace and Conversation Structure. It's designed for multi-turn, chat-style agents.

If clustering isn't set up yet, the panel shows an Enable clustering button. Turning it on sets up both views.

Intent: what users are asking

Clustering analyzes conversations, works out what your end users are asking about, names each intent it finds, and sorts every new conversation into one as it arrives. No tagging, no instrumentation, no taxonomy to maintain.

Each cluster includes the intent, its share of conversations, a description of what it covers, and how many conversations are in it. Click a card to filter the list to those conversations. The conversation table also has an Intent cluster column, so you can see where each conversation landed without filtering at all.

Custom intent clusters: track what matters to you

Custom clusters let you define clusters of your own, so anything you specifically want to watch gets its own grouping: a feature you just launched, a compliance-sensitive request type, an escalation path. They sit alongside the discovered ones and are classified the same way.

Creating, editing, and deleting them requires monitor write permission.

A custom cluster has a name and a description. The description is key: it's what conversations get matched against. Be specific about the requests you want caught, the way you'd describe them to a colleague. A description like Billing questions will pull in anything money-adjacent; one like this won't:

Users asking to close, freeze, or permanently delete their account, including requests routed through a support escalation.

You can create up to 10 custom clusters per agent, and they appear at the top of the list with a Custom badge.

Editing a cluster changes what happens next, not what already happened. Reword a description and future conversations get matched against the new wording; conversations already sorted stay where they are. To re-sort history under a new definition, delete the cluster and create it again.

Issues: where the agent is failing

Issues clusters show which conversations went poorly, whatever the user was asking for. These clusters are pre-defined:

IssueWhat it catches
Long-runningConversations in the slowest 5% of recent traffic
Token-heavyConversations in the most token-hungry 5% of recent traffic
Repeated tool callsConversations where a single tool was called more than 10 times, usually a loop
Low evaluation scoreConversations your evaluation monitors scored below the quality bar, one cluster per dimension being scored

The line above the cards tells you how much of your traffic is affected: 209 of 3,102 conversations carry at least one issue (6%).

A conversation can be slow and token-heavy and stuck in a tool loop, so it shows up in all three. The Issues column lists every issue a conversation hit; the ones carrying three at once are usually where to start.

Long-running and Token-heavy are relative. They flag the worst conversations for this agent, recently, not against a fixed target. So roughly 5% of your traffic sits in each one even on a healthy agent. Seeing them populated is not an incident: it's a queue of the worst conversations to review. And as the agent gets faster or slower overall, the bar moves with it. To catch latency crossing a threshold, create an Agent Metric Monitor.

Low evaluation score clusters come from Agent Evaluation Monitors scoped to evaluate entire conversations, one per dimension being scored. A monitor checking both helpfulness and completeness gives you two clusters. They appear once that evaluation starts producing scores.

FAQ

The Clusters panel is empty. Is clustering not enabled?

Not necessarily โ€” check whether you've selected a workflow first.

An agent running several workflows is clustered per workflow. Use the workflow filter on the Conversations tab and that workflow's clusters appear. Each workflow has its own topic set.

If you've selected a workflow and still see an Enable clustering button, clustering isn't on for that agent yet. Go ahead and enable it.

Long-running always sits at about 5%. Is that bad?

No โ€” that's the resting state, because the threshold is relative to this agent's own recent traffic.

Related


Did this page help you?