Agent Observability
Understand how Agents are being used.
- Available on the Team and Enterprise plans.
- Users need the Admin or Manager role to access the observability dashboard.
- Only Admins can view individual conversations.
The Context Studio includes built-in observability tools that help you understand how agents are being used and how well they are performing. Instead of relying on anecdotal feedback, the Context Studio surfaces patterns across conversations so you can focus your efforts where they will have the most impact.
Dashboard
The Dashboard provides a high-level view of agent usage across your workspace. It is the starting point for identifying trends, recurring questions, and areas where agents may need additional context.
From the Dashboard, you can:
- Track conversation volume over time
- See common topics across agent conversations
- Review patterns in warnings and user feedback
- Filter conversations to narrow in on specific areas of interest
Filters
Use filters to narrow the Dashboard to specific conversations or usage patterns. You can filter by:
- Agent type (Threads, Notebook agent, Modeling agent, Published app agent)
- Workspace role
- Individual users
- Source location (Hex, Slack, or MCP)
- Topics
- Warnings
- Time range (7, 30, or 90 days)
Managers can view aggregated metrics and trends. Admins can also open individual conversations for deeper inspection.
Topics
Topics group agent conversations into meaningful categories based on shared patterns. They help you understand what kinds of questions users are asking across your workspace and which areas come up most often.
Topics are generated automatically and may not appear for every conversation. In some cases, a topic may be assigned after a conversation becomes inactive.
Warnings
Warnings are system-generated signals that identify moments in a conversation where additional context may be needed to improve agent responses. They highlight cases where an agent expressed uncertainty, relied on limited context, or required user clarification.
Warnings help guide context improvements by pointing to where additional guidance or data definitions may be useful.
Warnings appear on individual conversations and are also summarized on the Dashboard to help identify recurring patterns. When you open a conversation with a warning, you can review additional detail and suggested context updates in the Thread Inspector.
Thread Inspector
The Thread Inspector is a read-only view that allows Admins to understand what happened in a specific agent conversation. It provides detailed context for reviewing agent behavior and following up on Topics or Warnings.
The Thread Inspector includes three sections:
Overview
The Overview provides a high-level summary of the conversation, including who asked the question, when the conversation occurred, and a synopsis of the user’s questions and the agent’s responses.
Timeline
The Timeline shows how the conversation unfolded step by step, including the sequence of messages and the tools the agent used.
This view lets you review individual turns in the conversation, inspect tool calls made by the agent, and see the primary data assets referenced at each stage. It helps explain how the agent arrived at its response.
You can click into individual tool calls to jump to where they occurred in the conversation.
Warnings
The Warnings tab shows any warnings associated with this conversation. These warnings highlight potential gaps in context that may need to be addressed to improve future agent responses.
Each warning includes suggested context updates to help improve future agent responses.
Next steps
Use what you learn from agent conversations to review and refine the context that shapes future responses.
- Context Management: Update guides, endorsements, and semantic models