Client events

Client events are system-level events sent from the server to the client that facilitate real-time communication. These events deliver audio, transcription, agent responses, and other critical information to the client application.

Overview

Client events are essential for maintaining the real-time nature of conversations. They provide everything from initialization metadata to processed audio and agent responses.

Client event types

conversation_initiation_metadata

Automatically sent when starting a conversation
Initializes conversation settings and parameters

JSON

// Example initialization metadata
{
  "type": "conversation_initiation_metadata",
  "conversation_initiation_metadata_event": {
    "conversation_id": "conv_123",
    "agent_output_audio_format": "pcm_44100",  // TTS output format
    "user_input_audio_format": "pcm_16000"    // ASR input format
  }
}

queue_status

Sent only to callers held in the call queue while the agent is at its concurrency limit
waiting is sent once, after conversation_initiation_metadata and before any hold audio
admitted or timed_out is sent once when the wait ends. timed_out is followed by a WebSocket close with code 4300
Always sent to queued callers. It does not need to be enabled in the agent's client_events configuration

JSON

// Example queue status event structure
{
  "type": "queue_status",
  "queue_status_event": {
    "status": "waiting"  // "waiting" | "admitted" | "timed_out"
  }
}

JavaScript

// Example queue status handler
websocket.on('queue_status', (event) => {
  const { status } = event.queue_status_event;
  if (status === 'waiting') {
    showWaitingState();
  } else if (status === 'admitted') {
    hideWaitingState();
  } else if (status === 'timed_out') {
    showAllAgentsBusyMessage();
  }
});

ping

Health check event requiring immediate response
Automatically handled by SDK
Used to maintain WebSocket connection

JSON

  // Example ping event structure
  {
    "ping_event": {
      "event_id": 123456,
      "ping_ms": 50  // Optional, estimated latency in milliseconds
    },
    "type": "ping"
  }

JavaScript

  // Example ping handler
  websocket.on('ping', () => {
    websocket.send('pong');
  });

audio

Contains base64 encoded audio for playback
Includes numeric event ID for tracking and sequencing
Handles voice output streaming
Includes alignment data with character-level timing information

JSON

// Example audio event structure
{
  "audio_event": {
    "audio_base_64": "base64_encoded_audio_string",
    "event_id": 12345,
    "alignment": {  // Character-level timing data
      "chars": ["H", "e", "l", "l", "o"],
      "char_durations_ms": [50, 30, 40, 40, 60],
      "char_start_times_ms": [0, 50, 80, 120, 160]
    }
  },
  "type": "audio"
}

JavaScript

// Example audio event handler
websocket.on('audio', (event) => {
  const { audio_event } = event;
  const { audio_base_64, event_id, alignment } = audio_event;
  audioPlayer.play(audio_base_64);

  // Use alignment data for synchronized text display
  const { chars, char_start_times_ms } = alignment;
  chars.forEach((char, i) => {
    setTimeout(() => highlightCharacter(char, i), char_start_times_ms[i]);
  });
});

user_transcript

Contains finalized speech-to-text results
Represents complete user utterances
Used for conversation history

JSON

// Example transcript event structure
{
  "type": "user_transcript",
  "user_transcription_event": {
    "user_transcript": "Hello, how can you help me today?"
  }
}

JavaScript

// Example transcript handler
websocket.on('user_transcript', (event) => {
  const { user_transcription_event } = event;
  const { user_transcript } = user_transcription_event;
  updateConversationHistory(user_transcript);
});

agent_response

Contains the complete agent message
Sent once that message is finished, so in voice conversations it usually arrives after the message's audio has already started streaming.
Used for display and history

JSON

// Example response event structure
{
  "type": "agent_response",
  "agent_response_event": {
    "agent_response": "Hello, how can I assist you today?"
  }
}

JavaScript

// Example response handler
websocket.on('agent_response', (event) => {
  const { agent_response_event } = event;
  const { agent_response } = agent_response_event;
  displayAgentMessage(agent_response);
});

agent_response_correction

Contains truncated response after interruption
Updates displayed message
Maintains conversation accuracy

JSON

// Example response correction event structure
{
  "type": "agent_response_correction",
  "agent_response_correction_event": {
    "original_agent_response": "Let me tell you about the complete history...",
    "corrected_agent_response": "Let me tell you about..."  // Truncated after interruption
  }
}

JavaScript

// Example response correction handler
websocket.on('agent_response_correction', (event) => {
  const { agent_response_correction_event } = event;
  const { corrected_agent_response } = agent_response_correction_event;
  displayAgentMessage(corrected_agent_response);
});

agent_response_metadata

Contains arbitrary metadata from a custom LLM response
Only sent when using a custom LLM
Must be explicitly enabled in the agent's client_events configuration

JSON

// Example agent response metadata event structure
{
  "type": "agent_response_metadata",
  "agent_response_metadata_event": {
    "metadata": {
      // Any key-value pairs returned by your custom LLM
      "key": "value"
    },
    "event_id": 12345
  }
}

JavaScript

// Example metadata handler
websocket.on('agent_response_metadata', (event) => {
  const { agent_response_metadata_event } = event;
  const { metadata, event_id } = agent_response_metadata_event;

  // Use metadata for UI updates, logging, or analytics
  console.log(`Response ${event_id} metadata:`, metadata);
  updateResponseDetails(metadata);
});

client_tool_call

Represents a function call the agent wants the client to execute
Contains tool name, tool call ID, and parameters
Requires client-side execution of the function and sending the result back to the server

JSON

// Example tool call event structure
{
  "type": "client_tool_call",
  "client_tool_call": {
    "tool_name": "search_database",
    "tool_call_id": "call_123456",
    "parameters": {
      "query": "user information",
      "filters": {
        "date": "2024-01-01"
      }
    }
  }
}

JavaScript

// Example tool call handler
websocket.on('client_tool_call', async (event) => {
  const { client_tool_call } = event;
  const { tool_name, tool_call_id, parameters } = client_tool_call;

  try {
    const result = await executeClientTool(tool_name, parameters);
    // Send success response back to continue conversation
    websocket.send({
      type: "client_tool_result",
      tool_call_id: tool_call_id,
      result: result,
      is_error: false
    });
  } catch (error) {
    // Send error response if tool execution fails
    websocket.send({
      type: "client_tool_result",
      tool_call_id: tool_call_id,
      result: error.message,
      is_error: true
    });
  }
});

agent_tool_response

Indicates when the agent has executed a tool function
Contains tool metadata and execution status
Provides visibility into agent tool usage during conversations

JSON

// Example agent tool response event structure
{
  "type": "agent_tool_response",
  "agent_tool_response": {
    "tool_name": "skip_turn",
    "tool_call_id": "skip_turn_c82ca55355c840bab193effb9a7e8101",
    "tool_type": "system",
    "is_error": false
  }
}

JavaScript

// Example agent tool response handler
websocket.on('agent_tool_response', (event) => {
  const { agent_tool_response } = event;
  const { tool_name, tool_call_id, tool_type, is_error } = agent_tool_response;

  if (is_error) {
    console.error(`Agent tool ${tool_name} failed:`, tool_call_id);
  } else {
    console.log(`Agent executed ${tool_type} tool: ${tool_name}`);
  }
});

agent_tool_response_full_payload

Mirrors agent_tool_response and additionally streams the tool's full result payload as a string in full_tool_result.
Surfaces tool output in the client for display or downstream processing.
Must be explicitly enabled in the agent's client_events configuration.

JSON

// Example agent tool response full payload event structure
{
  "type": "agent_tool_response_full_payload",
  "agent_tool_response_full_payload": {
    "tool_name": "lookup_order",
    "tool_call_id": "lookup_order_c82ca55355c840bab193effb9a7e8101",
    "tool_type": "webhook",
    "is_error": false,
    "full_tool_result": "{\"order_id\": \"ORD-789\", \"status\": \"shipped\"}",
    "truncated": false
  }
}

React

TSX

// Example agent tool response full payload handler (using @elevenlabs/react)
import { ConversationProvider } from '@elevenlabs/react';

function App() {
  return (
    <ConversationProvider
      onAgentToolResponse={(response) => {
        if (!('full_tool_result' in response)) return;
        const { tool_name, tool_call_id, is_error, full_tool_result, truncated } = response;

        if (is_error) {
          console.error(`Agent tool ${tool_name} failed:`, tool_call_id);
        } else {
          console.log(`Tool ${tool_name} returned:`, full_tool_result);
        }

        if (truncated) {
          console.warn(`Tool ${tool_name} result was truncated (exceeded 64 KB).`);
        }
      }}
    >
      <Agent />
    </ConversationProvider>
  );
}

JavaScript

// Example agent tool response full payload handler (using @elevenlabs/client)
import { Conversation } from '@elevenlabs/client';

const conversation = await Conversation.startSession({
  agentId: 'agent_7101k5zvyjhmfg983brhmhkd98n6',
  onAgentToolResponse: (response) => {
    if (!('full_tool_result' in response)) return;
    const { tool_name, tool_call_id, is_error, full_tool_result, truncated } = response;

    if (is_error) {
      console.error(`Agent tool ${tool_name} failed:`, tool_call_id);
    } else {
      console.log(`Tool ${tool_name} returned:`, full_tool_result);
    }

    if (truncated) {
      console.warn(`Tool ${tool_name} result was truncated (exceeded 64 KB).`);
    }
  },
});

vad_score

Voice Activity Detection score event
Indicates the probability that the user is speaking
Values range from 0 to 1, where higher values indicate higher confidence of speech

JSON

// Example VAD score event
{
  "type": "vad_score",
  "vad_score_event": {
    "vad_score": 0.95
  }
}

mcp_tool_call

Indicates when the agent has executed a MCP tool function
Contains tool name, tool call ID, and parameters
Called with one of four states: loading, awaiting_approval, success and failure.

JSON

{
  "type": "mcp_tool_call",
  "mcp_tool_call": {
    "service_id": "xJ8kP2nQ7sL9mW4vR6tY",
    "tool_call_id": "call_123456",
    "tool_name": "search_database",
    "tool_description": "Search the database for user information",
    "parameters": {
      "query": "user information",
    },
    "timestamp": "2024-09-30T14:23:45.123456+00:00",
    "state": "loading",
    "approval_timeout_secs": 10
  }
}

agent_chat_response_part

Streams the agent's response text as it is generated, as start, delta and stop messages
Always sent in text-only mode; in voice conversations it must be explicitly enabled in the agent's client_events configuration
Not sent while the agent or an active procedure uses a blocking guardrail, which has to evaluate the whole response before any of it is released
response_id identifies the message being streamed and matches the response_id of the agent_response that later commits it

JSON

// Example start event
{
  "type": "agent_chat_response_part",
  "text_response_part": {
    "type": "start",
    "text": "",
    "event_id": 12345,
    "response_id": "f47ac10b-58cc-4372-a567-0e02b2c3d479"
  }
}

JSON

// Example delta event with text chunk
{
  "type": "agent_chat_response_part",
  "text_response_part": {
    "type": "delta",
    "text": "Hello, how can I",
    "event_id": 12345,
    "response_id": "f47ac10b-58cc-4372-a567-0e02b2c3d479"
  }
}

JSON

// Example stop event
{
  "type": "agent_chat_response_part",
  "text_response_part": {
    "type": "stop",
    "text": "",
    "event_id": 12345,
    "response_id": "f47ac10b-58cc-4372-a567-0e02b2c3d479"
  }
}

JavaScript

// Example handler
websocket.on('agent_chat_response_part', (event) => {
  const { text_response_part } = event;
  const { type: partType, text, response_id } = text_response_part;

  if (partType === 'start') {
    initializeResponseBuffer(response_id);
  } else if (partType === 'delta') {
    appendToResponseBuffer(response_id, text);
  } else if (partType === 'stop') {
    finalizeResponse(response_id);
  }
});

agent_reasoning_response_part

agent_reasoning_response_part streams model-provided reasoning during text-only conversations. Enable the event in client_events and turn on Reasoning summary for the agent. The server sends start, delta, and stop messages. It does not send this event during voice conversations or while the agent or an active procedure uses blocking guardrails.

Event payload

Event payload

{  "type": "agent_reasoning_response_part",  "reasoning_response_part": {    "type": "delta",    "text": "The user asked to cancel, so I should verify the account before continuing.",    "event_id": 123456  }}

Start and stop events use an empty text value.

Handle reasoning events

Handle reasoning events

import { Conversation } from '@elevenlabs/client';const conversation = await Conversation.startSession({  agentId: 'agent_7101k5zvyjhmfg983brhmhkd98n6',  textOnly: true,  onAgentReasoningResponsePart: ({ type, text, event_id }) => {    if (type === 'start') {      initializeReasoningBuffer(event_id);    } else if (type === 'delta') {      appendToReasoningBuffer(text);    } else if (type === 'stop') {      finalizeReasoning();    }  },});

agent_response_complete

Fires when the agent has finished its response, including any pending tool calls. After this event, the agent will only produce further output if the user provides new input or a turn timeout triggers a new turn.
Must be explicitly enabled in the agent's client_events configuration

JSON

// Example agent response complete event structure
{
  "type": "agent_response_complete",
  "agent_response_complete_event": {
    "event_id": 12345
  }
}

JavaScript

// Example handler
websocket.on('agent_response_complete', (event) => {
  const { agent_response_complete_event } = event;
  const { event_id } = agent_response_complete_event;

  console.log(`Agent response ${event_id} complete`);
});

guardrail_triggered

Fires when a guardrail violation ends the conversation. Not sent when a guardrail triggers a retry that succeeds.
The event itself is the signal - it carries no payload beyond the type field.
Must be explicitly enabled in the agent's client_events configuration.

JSON

// Example guardrail triggered event structure
{
  "type": "guardrail_triggered"
}

JavaScript

// Example guardrail triggered handler (using @elevenlabs/client)
import { Conversation } from '@elevenlabs/client';

const conversation = await Conversation.startSession({
  agentId: 'agent_7101k5zvyjhmfg983brhmhkd98n6',
  onGuardrailTriggered: () => {
    console.warn('Guardrail triggered — conversation will end.');
  },
});

Event flow

Here's a typical sequence of events during a conversation:

sequenceDiagram
    participant Client
    participant Server

    Server->>Client: conversation_initiation_metadata
    Note over Client,Server: Connection established
    Server->>Client: ping
    Client->>Server: pong
    Server->>Client: audio
    Note over Client: Playing audio
    Note over Client: User responds
    Server->>Client: user_transcript
    Server->>Client: audio
    Server->>Client: agent_response
    Server->>Client: client_tool_call
    Note over Client: Client tool runs
    Client->>Server: client_tool_result
    Server->>Client: audio
    Server->>Client: agent_response
    Note over Client: Playing audio
    Note over Client: Interruption detected
    Server->>Client: agent_response_correction

When an agent is at its concurrency limit and call queueing is enabled, the server sends queue_status events between conversation_initiation_metadata and the first audio event. Hold audio is delivered as audio events until the caller is admitted.

Best practices

Error handling
- Implement proper error handling for each event type
- Log important events for debugging
- Handle connection interruptions gracefully
Audio management
- Buffer audio chunks appropriately
- Implement proper cleanup on interruption
- Handle audio resource management
Connection management
- Respond to PING events promptly
- Implement reconnection logic
- Monitor connection health

Troubleshooting

Connection issues

Ensure proper WebSocket connection
Check PING/PONG responses
Verify API credentials

Audio problems

Check audio chunk handling
Verify audio format compatibility
Monitor memory usage

Event handling

Log all events for debugging
Implement error boundaries
Check event handler registration

Search documentation

Help Center

Client events

Overview

Client event types

conversation_initiation_metadata

queue_status

ping

audio

user_transcript

agent_response

agent_response_correction

agent_response_metadata

client_tool_call

agent_tool_response

agent_tool_response_full_payload

React

JavaScript

vad_score

mcp_tool_call

agent_chat_response_part

agent_reasoning_response_part

agent_response_complete

guardrail_triggered

Event flow

Best practices

Troubleshooting

Connection issues

Audio problems

Event handling

Help Center