Vite (Javascript)

Learn how to create a web application that enables voice conversations with ElevenLabs AI agents

This tutorial will guide you through creating a web client that can interact with a ElevenLabs agent. You’ll learn how to implement real-time voice conversations, allowing users to speak with an AI agent that can listen, understand, and respond naturally using voice synthesis.

What You’ll Need

An ElevenLabs agent created following this guide
npm installed on your local system
Basic knowledge of JavaScript

Project Setup

Create a Project Directory
Open a terminal and create a new directory for your project:
Bash
```
mkdir elevenlabs-conversational-ai
cd elevenlabs-conversational-ai
```
Initialize npm and Install Dependencies
Initialize a new npm project and install the required packages:
Bash
```
npm init -y
npm install vite @elevenlabs/client
```

Set up Basic Project Structure

Add this to your package.json:

package.json

{
    "scripts": {
        ...
        "dev:frontend": "vite"
    }
}

Create the following file structure:

Bash

elevenlabs-conversational-ai/
├── index.html
├── script.js
├── package-lock.json
├── package.json
└── node_modules

Implementing the Voice Chat Interface

Create the HTML Interface

In index.html, set up a simple user interface:

index.html

<!DOCTYPE html>
<html lang="en">
    <head>
        <meta charset="UTF-8" />
        <meta name="viewport" content="width=device-width, initial-scale=1.0" />
        <title>ElevenLabs Agents</title>
    </head>
    <body style="font-family: Arial, sans-serif; text-align: center; padding: 50px;">
        <h1>ElevenLabs Agents</h1>
        <div style="margin-bottom: 20px;">
            <button id="startButton" style="padding: 10px 20px; margin: 5px;">Start Conversation</button>
            <button id="stopButton" style="padding: 10px 20px; margin: 5px;" disabled>Stop Conversation</button>
        </div>
        <div style="font-size: 18px;">
            <p>Status: <span id="connectionStatus">Disconnected</span></p>
            <p>Agent is <span id="agentStatus">listening</span></p>
        </div>
        <script type="module" src="../images/script.js"></script>
    </body>
</html>

Implement the Conversation Logic

In script.js, implement the functionality:

script.js

import { Conversation } from '@elevenlabs/client';

const startButton = document.getElementById('startButton');
const stopButton = document.getElementById('stopButton');
const connectionStatus = document.getElementById('connectionStatus');
const agentStatus = document.getElementById('agentStatus');

let conversation;

async function startConversation() {
    try {
        // Request microphone permission
        await navigator.mediaDevices.getUserMedia({ audio: true });

        // Start the conversation
        conversation = await Conversation.startSession({
            agentId: 'YOUR_AGENT_ID', // Replace with your agent ID
            onConnect: () => {
                connectionStatus.textContent = 'Connected';
                startButton.disabled = true;
                stopButton.disabled = false;
            },
            onDisconnect: () => {
                connectionStatus.textContent = 'Disconnected';
                startButton.disabled = false;
                stopButton.disabled = true;
            },
            onError: (error) => {
                console.error('Error:', error);
            },
            onModeChange: (mode) => {
                agentStatus.textContent = mode.mode === 'speaking' ? 'speaking' : 'listening';
            },
        });
    } catch (error) {
        console.error('Failed to start conversation:', error);
    }
}

async function stopConversation() {
    if (conversation) {
        await conversation.endSession();
        conversation = null;
    }
}

startButton.addEventListener('click', startConversation);
stopButton.addEventListener('click', stopConversation);

Start the frontend server
Bash
```
npm run dev:frontend
```

(Optional) Authenticate with a Signed URL

Create Environment Variables
Create a .env file in your project root:
title=".env"
```
ELEVENLABS_API_KEY=your-api-key-here
AGENT_ID=your-agent-id-here
```
Make sure to add .env to your .gitignore file to prevent accidentally committing sensitive credentials.
Setup the Backend
1. Install additional dependencies:
Bash
```
npm install express cors dotenv
```
1. Create a new folder called backend:
Bash
```
elevenlabs-conversational-ai/
├── backend
...
```

Create the Server

backend/server.js

require("dotenv").config();

const express = require("express");
const cors = require("cors");

const app = express();
app.use(cors());
app.use(express.json());

const PORT = process.env.PORT || 3001;

app.get("/api/get-signed-url", async (req, res) => {
    try {
        const response = await fetch(
            `https://api.elevenlabs.io/v1/convai/conversation/get-signed-url?agent_id=${process.env.AGENT_ID}`,
            {
                headers: {
                    "xi-api-key": process.env.ELEVENLABS_API_KEY,
                },
            }
        );

        if (!response.ok) {
            throw new Error("Failed to get signed URL");
        }

        const data = await response.json();
        res.json({ signedUrl: data.signed_url });
    } catch (error) {
        console.error("Error:", error);
        res.status(500).json({ error: "Failed to generate signed URL" });
    }
});

app.listen(PORT, () => {
    console.log(`Server running on http://localhost:${PORT}`);
});

Update the Client Code

Modify your script.js to fetch and use the signed URL:

script.js

// ... existing imports and variables ...

async function getSignedUrl() {
    const response = await fetch('http://localhost:3001/api/get-signed-url');
    if (!response.ok) {
        throw new Error(`Failed to get signed url: ${response.statusText}`);
    }
    const { signedUrl } = await response.json();
    return signedUrl;
}

async function startConversation() {
    try {
        await navigator.mediaDevices.getUserMedia({ audio: true });

        const signedUrl = await getSignedUrl();

        conversation = await Conversation.startSession({
            signedUrl,
            // agentId has been removed...
            onConnect: () => {
                connectionStatus.textContent = 'Connected';
                startButton.disabled = true;
                stopButton.disabled = false;
            },
            onDisconnect: () => {
                connectionStatus.textContent = 'Disconnected';
                startButton.disabled = false;
                stopButton.disabled = true;
            },
            onError: (error) => {
                console.error('Error:', error);
            },
            onModeChange: (mode) => {
                agentStatus.textContent = mode.mode === 'speaking' ? 'speaking' : 'listening';
            },
        });
    } catch (error) {
        console.error('Failed to start conversation:', error);
    }
}

// ... rest of the code ...

Update the package.json

package.json

{
    "scripts": {
        ...
        "dev:backend": "node backend/server.js",
        "dev": "npm run dev:frontend & npm run dev:backend"
    }
}

Run the Application
Start the application with:
Bash
```
npm run dev
```

Next Steps

Now that you have a basic implementation, you can:

Add visual feedback for voice activity
Implement error handling and retry logic
Add a chat history display
Customize the UI to match your brand

Search documentation

Help Center

Vite (Javascript)

What You’ll Need

Project Setup

Implementing the Voice Chat Interface

Next Steps

Help Center