Kotlin SDK
ElevenAgents SDK: deploy customized, interactive voice agents in minutes for Android apps.
Installation
Section titled “Installation”Add the ElevenLabs SDK to your Android project by including the following dependency in your app-level build.gradle file:
dependencies {
// ElevenLabs Agents SDK (Android)
implementation("io.elevenlabs:elevenlabs-android:<latest>")
// Kotlin coroutines, AndroidX, etc., as needed by your app
}Requirements
Section titled “Requirements”- Android API level 21 (Android 5.0) or higher
- Internet permission for API calls
- Microphone permission for voice input
- Network security configuration for HTTPS calls
Manifest Configuration
Section titled “Manifest Configuration”Add the necessary permissions to your AndroidManifest.xml:
<uses-permission android:name="android.permission.INTERNET" />
<uses-permission android:name="android.permission.RECORD_AUDIO" />
<uses-permission android:name="android.permission.MODIFY_AUDIO_SETTINGS" />Runtime Permissions
Section titled “Runtime Permissions”For Android 6.0 (API level 23) and higher, you must request microphone permission at runtime:
import android.Manifest
import android.content.pm.PackageManager
import androidx.core.app.ActivityCompat
import androidx.core.content.ContextCompat
private fun requestMicrophonePermission() {
if (ContextCompat.checkSelfPermission(this, Manifest.permission.RECORD_AUDIO)
!= PackageManager.PERMISSION_GRANTED) {
if (ActivityCompat.shouldShowRequestPermissionRationale(this, Manifest.permission.RECORD_AUDIO)) {
// Show explanation to the user
showPermissionExplanationDialog()
} else {
ActivityCompat.requestPermissions(
this,
arrayOf(Manifest.permission.RECORD_AUDIO),
MICROPHONE_PERMISSION_REQUEST_CODE
)
}
}
}Initialize the ElevenLabs SDK in your Application class or main activity:
Start a conversation session with either:
- Public agent: pass
agentId - Private agent: pass
conversationTokenprovisioned from your backend (never expose your API key to the client).
import io.elevenlabs.ConversationClient
import io.elevenlabs.ConversationConfig
import io.elevenlabs.ConversationSession
import io.elevenlabs.ClientTool
import io.elevenlabs.ClientToolResult
// Start a public agent session (token generated for you)
val config = ConversationConfig(
agentId = "<your_public_agent_id>", // OR conversationToken = "<token>"
userId = "your-user-id",
// Optional callbacks
onConnect = { conversationId ->
// Called when the conversation is connected and returns the conversation ID. You can access conversationId via session.getId() too
},
onMessage = { source, messageJson ->
// Raw JSON messages from data channel; useful for logging/telemetry
},
onModeChange = { mode ->
// "speaking" | "listening" — drive UI indicators
},
onStatusChange = { status ->
// "connected" | "connecting" | "disconnected"
},
onCanSendFeedbackChange = { canSend ->
// Enable/disable thumbs up/down buttons for feedback reporting
},
onUnhandledClientToolCall = { call ->
// Agent requested a client tool not registered on the device
},
onVadScore = { score ->
// Voice Activity Detection score, range from 0 to 1 where higher values indicate higher confidence of speech
},
onAudioAlignment = { alignment ->
// Character-level timing data for synchronized text display
val chars = alignment["chars"] as? List<*>
val startTimes = alignment["char_start_times_ms"] as? List<*>
val durations = alignment["char_durations_ms"] as? List<*>
Log.d("ExampleApp", "Audio alignment: $chars")
},
// List of client tools the agent can invoke
clientTools = mapOf(
"logMessage" to object : ClientTool {
override suspend fun execute(parameters: Map<String, Any>): ClientToolResult {
val message = parameters["message"] as? String
Log.d("ExampleApp", "[INFO] Client Tool Log: $message")
return ClientToolResult.success("Message logged successfully")
}
}
),
)
// In an Activity context
val session: ConversationSession = ConversationClient.startSession(config, this)Note that ElevenAgents requires microphone access. Consider explaining and requesting permissions in your app’s UI before the conversation starts, especially on Android 6.0+ where runtime permissions are required.
Public vs Private Agents
Section titled “Public vs Private Agents”- Public agents (no auth): Initialize with
agentIdinConversationConfig. The SDK requests a conversation token from ElevenLabs without needing an API key on device. - Private agents (auth): Initialize with
conversationTokeninConversationConfig. Your server requests a conversation token from ElevenLabs using your ElevenLabs API key.
Client Tools
Section titled “Client Tools”Register client tools to allow the agent to call local capabilities on the device.
val config = ConversationConfig(
agentId = "<public_agent>",
clientTools = mapOf(
"logMessage" to object : io.elevenlabs.ClientTool {
override suspend fun execute(parameters: Map<String, Any>): io.elevenlabs.ClientToolResult? {
val message = parameters["message"] as? String ?: return io.elevenlabs.ClientToolResult.failure("Missing 'message'")
android.util.Log.d("ClientTool", "Log: $message")
return null // No response needed for fire-and-forget tools
}
}
)
)When the agent issues a client_tool_call, the SDK executes the matching tool and responds with a client_tool_result. If the tool is not registered, onUnhandledClientToolCall is invoked and a failure result is returned to the agent (if a response is expected).
Callbacks Overview
Section titled “Callbacks Overview”- onConnect - Called when the WebRTC connection is established. Returns the conversation ID.
- onMessage - Called when a new message is received. These can be tentative or final transcriptions of user voice, replies produced by LLM, or debug messages. Provides source (
"ai"or"user") and raw JSON message. - onModeChange - Called when the conversation mode changes. This is useful for indicating whether the agent is speaking (
"speaking") or listening ("listening"). - onStatusChange - Called when the conversation status changes (
"connected","connecting", or"disconnected"). - onCanSendFeedbackChange - Called when the ability to send feedback changes. Enables/disables feedback buttons.
- onUnhandledClientToolCall - Called when the agent requests a client tool that is not registered on the device.
- onVadScore - Called when the voice activity detection score changes. Range from 0 to 1 where higher values indicate higher confidence of speech.
- onAudioAlignment - Called when audio alignment data is received, providing character-level timing information for agent speech.
Methods
Section titled “Methods”startSession
Section titled “startSession”The startSession method initiates the WebRTC connection and starts using the microphone to communicate with the ElevenLabs Agents agent.
Public agents
Section titled “Public agents”For public agents (i.e. agents that don’t have authentication enabled), only the agentId is required. The Agent ID can be acquired through the ElevenLabs UI.
val session = ConversationClient.startSession(
config = ConversationConfig(
agentId = "your-agent-id"
),
context = this
)Private agents
Section titled “Private agents”For private agents, you must pass in a conversationToken obtained from the ElevenLabs API. Generating this token requires an ElevenLabs API key.
// Server-side token generation (Node.js example)
app.get("/conversation-token", yourAuthMiddleware, async (req, res) => {
const response = await fetch(
`https://api.elevenlabs.io/v1/convai/conversation/token?agent_id=${process.env.AGENT_ID}`,
{
headers: {
// Requesting a conversation token requires your ElevenLabs API key
// Do NOT expose your API key to the client!
"xi-api-key": process.env.ELEVENLABS_API_KEY,
},
}
);
if (!response.ok) {
return res.status(500).send("Failed to get conversation token");
}
const body = await response.json();
res.send(body.token);
});Then, pass the token to the startSession method. Note that only the conversationToken is required for private agents.
// Get conversation token from your server
val conversationToken = fetchConversationTokenFromServer()
// For private agents, pass in the conversation token
val session = ConversationClient.startSession(
config = ConversationConfig(
conversationToken = conversationToken
),
context = this
)You can optionally pass a user ID to identify the user in the conversation. This can be your own customer identifier. This will be included in the conversation initiation data sent to the server.
val session = ConversationClient.startSession(
config = ConversationConfig(
agentId = "your-agent-id",
userId = "your-user-id"
),
context = this
)endSession
Section titled “endSession”A method to manually end the conversation. The method will disconnect and end the conversation.
session.endSession()sendUserMessage
Section titled “sendUserMessage”Send a text message to the agent during an active conversation. This will trigger a response from the agent.
session.sendUserMessage("Hello, how can you help me?")sendContextualUpdate
Section titled “sendContextualUpdate”Sends contextual information to the agent that won’t trigger a response.
session.sendContextualUpdate(
"User navigated to the profile page. Consider this for next response."
)sendFeedback
Section titled “sendFeedback”Provide feedback on the conversation quality. This helps improve the agent’s performance. Use onCanSendFeedbackChange to enable your thumbs up/down UI when feedback is allowed.
// Positive feedback
session.sendFeedback(true)
// Negative feedback
session.sendFeedback(false)sendUserActivity
Section titled “sendUserActivity”Notifies the agent about user activity to prevent interruptions. Useful for when the user is actively using the app and the agent should pause speaking, i.e. when the user is typing in a chat.
The agent will pause speaking for ~2 seconds after receiving this signal.
session.sendUserActivity()Get the conversation ID.
val conversationId = session.getId()
Log.d("Conversation", "Conversation ID: $conversationId")
// e.g., "conv_123"Mute/ Unmute
Section titled “Mute/ Unmute”session.toggleMute()
session.setMicMuted(true) // mute
session.setMicMuted(false) // unmuteObserve session.isMuted to update the UI label between “Mute” and “Unmute”.
Properties
Section titled “Properties”status
Section titled “status”Get the current status of the conversation.
val status = session.status
Log.d("Conversation", "Current status: $status")
// Values: DISCONNECTED, CONNECTING, CONNECTEDProGuard / R8
Section titled “ProGuard / R8”If you shrink/obfuscate, ensure Gson models and LiveKit are kept. Example rules (adjust as needed):
-keep class io.elevenlabs.** { *; }
-keep class io.livekit.** { *; }
-keepattributes *Annotation*Troubleshooting
Section titled “Troubleshooting”- Ensure microphone permission is granted at runtime
- If reconnect hangs, verify your app calls
session.endSession()and that you start a new session instance before reconnecting - For emulators, verify audio input/output routes are working; physical devices tend to behave more reliably
Example Implementation
Section titled “Example Implementation”For an example implementation, see the example app in the ElevenLabs Android SDK repository. The app demonstrates:
- One‑tap connect/disconnect
- Speaking/listening indicator
- Feedback buttons with UI enable/disable
- Typing indicator via
sendUserActivity() - Contextual and user messages from an input
- Microphone mute/unmute button