Client-side streaming
Client-side streaming
Overview
Section titled “Overview”The ElevenLabs Realtime Speech to Text API enables you to transcribe audio streams in real-time with ultra-low latency using the Scribe Realtime v2 model. Whether you’re building voice assistants, transcription services, or any application requiring live speech recognition, this WebSocket-based API delivers partial transcripts as you speak and committed transcripts when speech segments are complete.
Scribe v2 Realtime can be implemented on the client side to transcribe audio in realtime, either via the microphone or manually chunking the audio.
The client side implementation differs from server side in a few ways:
- Requires a single use token - this is a temporary token that can be used to connect to the API without exposing your API key.
- Audio from the microphone can be piped directly to the API to transcribe, without the need to manually chunk the audio.
For streaming audio from a URL, see the Server-side streaming guide.
Quickstart
Section titled “Quickstart”npm install @elevenlabs/react @elevenlabs/elevenlabs-jsnpm install @elevenlabs/client @elevenlabs/elevenlabs-jsimport { useScribe } from "@elevenlabs/react";
function MyComponent() {
const scribe = useScribe({
modelId: "scribe_v2_realtime",
onPartialTranscript: (data) => {
console.log("Partial:", data.text);
},
onCommittedTranscript: (data) => {
console.log("Committed:", data.text);
},
onCommittedTranscriptWithTimestamps: (data) => {
console.log("Committed with timestamps:", data.text);
console.log("Timestamps:", data.words);
},
});
const handleStart = async () => {
// Fetch a single use token from the server
const token = await fetchTokenFromServer();
await scribe.connect({
token,
microphone: {
echoCancellation: true,
noiseSuppression: true,
},
});
};
return (
<div>
<button onClick={handleStart} disabled={scribe.isConnected}>
Start Recording
</button>
<button onClick={scribe.disconnect} disabled={!scribe.isConnected}>
Stop
</button>
{scribe.partialTranscript && <p>Live: {scribe.partialTranscript}</p>}
<div>
{scribe.committedTranscripts.map((t) => (
<p key={t.id}>{t.text}</p>
))}
</div>
</div>
);
}// Client side
import { Scribe, RealtimeEvents } from "@elevenlabs/client";
// Ensure you have authentication headers set up
const response = await fetch("/scribe-token", yourAuthHeaders);
const { token } = await response.json();
const connection = Scribe.connect({
token,
modelId: "scribe_v2_realtime",
includeTimestamps: true,
microphone: {
echoCancellation: true,
noiseSuppression: true,
autoGainControl: true,
},
});
// Set up event handlers
// Session started
connection.on(RealtimeEvents.SESSION_STARTED, () => {
console.log("Session started");
});
// Partial transcripts (interim results), use this in your UI to show the live transcript
connection.on(RealtimeEvents.PARTIAL_TRANSCRIPT, (data) => {
console.log("Partial:", data.text);
});
// Committed transcripts
connection.on(RealtimeEvents.COMMITTED_TRANSCRIPT, (data) => {
console.log("Committed:", data.text);
});
// Committed transcripts with word-level timestamps. Only received when includeTimestamps is set to true.
connection.on(RealtimeEvents.COMMITTED_TRANSCRIPT_WITH_TIMESTAMPS, (data) => {
console.log("Committed:", data.text);
console.log("Timestamps:", data.words);
});
// Errors - will catch all errors, both server and websocket specific errors
connection.on(RealtimeEvents.ERROR, (error) => {
console.error("Error:", error);
});
// Connection opened
connection.on(RealtimeEvents.OPEN, () => {
console.log("Connection opened");
});
// Connection closed
connection.on(RealtimeEvents.CLOSE, () => {
console.log("Connection closed");
});
// When you are done, close the connection
connection.close();