Conversation-Level Agent Eval Monitors

Agent evaluation monitors can now score entire conversations, not just individual spans. For supported agents, the monitor builder includes a new scope toggle — Specific spans or Entire conversations — giving you the same configuration experience: eval templates or custom LLM-as-judge prompts, sampling controls, and the ability to test against real data before saving.

Many agent quality issues only surface at the conversation level — whether a user got their answer after several turns, whether the agent recovered from a bad tool call, or whether a session ended in frustration. This brings continuous, scheduled conversation-quality monitoring to multi-turn agents.