Operate

Monitor, test, and optimize your agents at scale.

This section provides tools and best practices for running production agents, from testing and quality assurance to analytics and cost optimization.

Goal	Guide	Description
List users by external ID	Users	See end users and open their conversations
Search transcripts	Searching conversations	Keyword and semantic search in Conversation history
Run A/B tests	Experiments	Test agent configuration changes with live traffic
Test agent behavior	Testing	Create and run automated tests for your agents
Analyze conversation quality	Conversation analysis	Extract insights and evaluate conversation outcomes
Track metrics & analytics	Analytics	Monitor performance metrics and conversation history
Configure data retention	Privacy	Set retention policies for conversations and audio
Reduce LLM costs	Cost optimization	Monitor and optimize language model expenses
Manage agents as code	CLI	Version control and deploy agents from your terminal
Manage agents from Claude	Hosted MCP	Create and update agents from any MCP client

Next steps

Users

List end users by ID and drill into their conversations

Experiments

Run A/B tests to optimize agent performance

Testing

Set up automated tests

Analytics

View performance metrics

Spotlight

Review real-time insights, sentiment, and discovered topics

Conversation analysis

Analyze conversations