Language detection

Overview

The language detection system tool allows your ElevenLabs agent to switch its output language to any the agent supports. This system tool is not enabled automatically. Its description can be customized to accommodate your specific use case.

Our language detection tool triggers language switching in two cases, both based on the received audio's detected language and content:

detection if a user speaks a different language than the current output language, a switch will be triggered
content if the user asks in the current language to change to a new language, a switch will be triggered

Purpose: Automatically switch to the user's detected language during conversations.

Trigger conditions: The LLM should call this tool when:

User speaks in a different language than the current conversation language
User explicitly requests to switch languages
Multi-language support is needed for the conversation

Parameters:

reason (string, required): The reason for the language switch
language (string, required): The language code to switch to (must be in supported languages list)

Function call format:

JSON

{
  "type": "function",
  "function": {
    "name": "language_detection",
    "arguments": "{\"reason\": \"User requested Spanish\", \"language\": \"es\"}"
  }
}

Implementation: Configure supported languages in agent settings and add the language detection system tool. The agent will automatically switch voice and responses to match detected languages.

Enabling language detection

Configure supported languages

The languages that the agent can switch to must be defined in the Agent settings tab.

Add the language detection tool

Enable language detection by selecting the pre-configured system tool to your agent's tools in the Agent tab. This is automatically available as an option when selecting add tool.

Configure tool description

Add a description that specifies when to call the tool

Restricting switching to the start of the conversation

In a long conversation, background speech or an unusual accent can occasionally trigger a switch the caller did not intend. Enabling only_at_conversation_start confines switching to the first two user turns.

If the language does not switch during those two turns, language detection is disabled for the rest of the conversation: every later attempt fails and the agent keeps speaking the current language. If the language does switch within that window, switching stays available for the rest of the conversation, so an incorrect detection can still be corrected.

The option is disabled by default.

Update via the dashboard

Select the Detect language tool in the Tools tab and enable Only at start of conversation.

Update via the CLI

Pull the agent configuration

Bash

elevenlabs agents pull

Edit `agent_configs/<agent-name>.json`

Set only_at_conversation_start on the language detection tool. Surrounding fields are omitted here for brevity.

JSON

{
  "conversation_config": {
    "agent": {
      "prompt": {
        "built_in_tools": {
          "language_detection": {
            "type": "system",
            "name": "language_detection",
            "params": {
              "system_tool_type": "language_detection",
              "only_at_conversation_start": true
            }
          }
        }
      }
    }
  }
}

Push your changes

Bash

elevenlabs agents push

Update via the API

Python

from elevenlabs import ElevenLabs

elevenlabs = ElevenLabs()

elevenlabs.conversational_ai.agents.update(
    agent_id="agent_7101k5zvyjhmfg983brhmhkd98n6",
    conversation_config={
        "agent": {
            "prompt": {
                "built_in_tools": {
                    "language_detection": {
                        "type": "system",
                        "name": "language_detection",
                        "params": {
                            "system_tool_type": "language_detection",
                            "only_at_conversation_start": True,
                        },
                    }
                }
            }
        }
    },
)

TypeScript

import { ElevenLabsClient } from "@elevenlabs/elevenlabs-js";

const elevenlabs = new ElevenLabsClient();

await elevenlabs.conversationalAi.agents.update("agent_7101k5zvyjhmfg983brhmhkd98n6", {
  conversationConfig: {
    agent: {
      prompt: {
        builtInTools: {
          languageDetection: {
            type: "system",
            name: "language_detection",
            params: {
              systemToolType: "language_detection",
              onlyAtConversationStart: true,
            },
          },
        },
      },
    },
  },
});

API Implementation

When creating an agent via API, you can add the language detection tool to your agent configuration. It should be defined as a system tool:

Python

from elevenlabs import AgentConfig, ConversationalConfig, ElevenLabs

elevenlabs = ElevenLabs(api_key="YOUR_API_KEY")

# Create language presets
language_presets = {
    "nl": {"overrides": {"agent": {"first_message": "Hoi, hoe gaat het met je?"}}},
    "fi": {"overrides": {"agent": {"first_message": "Hei, kuinka voit?"}}},
    "tr": {"overrides": {"agent": {"first_message": "Merhaba, nasılsın?"}}},
    "ru": {"overrides": {"agent": {"first_message": "Привет, как ты?"}}},
    "pt": {"overrides": {"agent": {"first_message": "Oi, como você está?"}}},
}

response = elevenlabs.conversational_ai.agents.create(
    conversation_config=ConversationalConfig(
        agent=AgentConfig(
            first_message="Hi how are you?",
            prompt={
                "built_in_tools": {
                    "language_detection": {
                        "type": "system",
                        "name": "language_detection",
                        # Optional: customize when the tool should be triggered
                        "description": "",
                        "params": {"system_tool_type": "language_detection"},
                    }
                },
            },
        ),
        language_presets=language_presets,
    ),
)

JavaScript

import { ElevenLabsClient } from "@elevenlabs/elevenlabs-js";

const elevenlabs = new ElevenLabsClient({
  apiKey: "YOUR_API_KEY",
});

await elevenlabs.conversationalAi.agents.create({
  conversationConfig: {
    agent: {
      firstMessage: "Hi, how are you?",
      prompt: {
        builtInTools: {
          languageDetection: {
            type: "system",
            name: "language_detection",
            description: "", // Optional: Customize when the tool should be triggered
            params: { systemToolType: "language_detection" },
          },
        },
      },
    },
    languagePresets: {
      nl: { overrides: { agent: { firstMessage: "Hoi, hoe gaat het met je?" } } },
      fi: {
        overrides: { agent: { firstMessage: "Hei, kuinka voit?" } },
        firstMessageTranslation: {
          sourceHash: '{"firstMessage":"Hi how are you?","language":"en"}',
          text: "Hei, kuinka voit?",
        },
      },
      tr: { overrides: { agent: { firstMessage: "Merhaba, nasılsın?" } } },
      ru: { overrides: { agent: { firstMessage: "Привет, как ты?" } } },
      pt: { overrides: { agent: { firstMessage: "Oi, como você está?" } } },
      ar: { overrides: { agent: { firstMessage: "مرحبًا كيف حالك؟" } } },
    },
  },
});

Bash

curl -X POST https://api.elevenlabs.io/v1/convai/agents/create \
     -H "xi-api-key: YOUR_API_KEY" \
     -H "Content-Type: application/json" \
     -d '{
  "conversation_config": {
    "agent": {
      "first_message": "Hi how are you?",
      "prompt": {
        "built_in_tools": {
          "language_detection": {
            "type": "system",
            "name": "language_detection",
            "description": "",
            "params": { "system_tool_type": "language_detection" }
          }
        }
      }
    },
    "language_presets": {
      "nl": {
        "overrides": {
          "agent": {
            "prompt": null,
            "first_message": "Hoi, hoe gaat het met je?",
            "language": null
          },
          "tts": null
        }
      },
      "fi": {
        "overrides": {
          "agent": {
            "prompt": null,
            "first_message": "Hei, kuinka voit?",
            "language": null
          },
          "tts": null
        }
      },
      "tr": {
        "overrides": {
          "agent": {
            "prompt": null,
            "first_message": "Merhaba, nasılsın?",
            "language": null
          },
          "tts": null
        }
      },
      "ru": {
        "overrides": {
          "agent": {
            "prompt": null,
            "first_message": "Привет, как ты?",
            "language": null
          },
          "tts": null
        }
      },
      "pt": {
        "overrides": {
          "agent": {
            "prompt": null,
            "first_message": "Oi, como você está?",
            "language": null
          },
          "tts": null
        }
      },
      "ar": {
        "overrides": {
          "agent": {
            "prompt": null,
            "first_message": "مرحبًا كيف حالك؟",
            "language": null
          },
          "tts": null
        }
      }
    }
  }
}'

Search documentation

Help Center

Language detection

Overview

Enabling language detection

Configure supported languages

Add the language detection tool

Configure tool description

Restricting switching to the start of the conversation

Update via the dashboard

Update via the CLI

Pull the agent configuration

Edit `agent_configs/<agent-name>.json`

Push your changes

Update via the API

API Implementation

Help Center