Voice

Last updated Oct 21, 2026 Build real-time voice agents with speech-to-text, text-to-speech, and conversation persistence. Audio streams over WebSocket — no SFU or meeting infrastructure required. Beta

Overview

@cloudflare/voice provides two server-side mixins and matching client libraries:

Built on Cloudflare Durable Objects, you get:

• Real-time audio — mic audio streams as binary WebSocket frames, TTS audio streams back
• Automatic conversation persistence — messages stored in SQLite, survive restarts
• Streaming TTS — LLM tokens are sentence-chunked and synthesized concurrently
• Interruption handling — user speech during playback cancels the current response
• Continuous STT — per-call transcriber session, model handles turn detection
• Pipeline hooks — intercept and transform text at every stage

Quick start

Install

npm install @cloudflare/voice agents

Server

import { Agent } from "agents";
import { withVoice, WorkersAIFluxSTT, WorkersAITTS } from "@cloudflare/voice";

const VoiceAgent = withVoice(Agent);

export class MyAgent extends VoiceAgent {
    transcriber = new WorkersAIFluxSTT(this.env.AI);
    tts = new WorkersAITTS(this.env.AI);

async onTurn(transcript, context) {
        return "Hello! I heard you say: " + transcript;
    }
}

import { Agent } from "agents";
import {
    withVoice,
    WorkersAIFluxSTT,
    WorkersAITTS,
    type VoiceTurnContext,
} from "@cloudflare/voice";

const VoiceAgent = withVoice(Agent);

export class MyAgent extends VoiceAgent<Env> {
    transcriber = new WorkersAIFluxSTT(this.env.AI);
    tts = new WorkersAITTS(this.env.AI);

async onTurn(transcript: string, context: VoiceTurnContext) {
        return "Hello! I heard you say: " + transcript;
    }
}

Client (React)

import { useVoiceAgent } from "@cloudflare/voice/react";

function VoiceUI() {
    const {
        status,
        transcript,
        interimTranscript,
        audioLevel,
        isMuted,
        startCall,
        endCall,
        toggleMute,
    } = useVoiceAgent({ agent: "MyAgent" });

return (
        <div>
            <p>Status: {status}</p>

<button onClick={status === "idle" ? startCall : endCall}>
                {status === "idle" ? "Start Call" : "End Call"}
            </button>

<button onClick={toggleMute}>{isMuted ? "Unmute" : "Mute"}</button>

{interimTranscript && (
                <p>
                    <em>{interimTranscript}</em>
                </p>
            )}

{transcript.map((msg, i) => (
                <p key={i}>
                    <strong>{msg.role}:</strong> {msg.text}
                </p>
            ))}
        </div>
    );
}

Wrangler configuration

{
    "ai": {
        "binding": "AI"
    },
    "durable_objects": {
        "bindings": [
            {
                "name": "MyAgent",
                "class_name": "MyAgent"
            }
        ]
    },
    "migrations": [
        {
            "tag": "v1",
            "new_sqlite_classes": ["MyAgent"]
        }
    ]
}

[ai]
binding = "AI"

[[durable_objects.bindings]]
name = "MyAgent"
class_name = "MyAgent"

[[migrations]]
tag = "v1"
new_sqlite_classes = [ "MyAgent" ]

How it works

Browser                              Durable Object (withVoice)
┌──────────┐                         ┌──────────────────────────┐
│ Mic      │   binary PCM (16kHz)    │ Transcriber session      │
│          │ ──────────────────────► │ (per-call, continuous)   │
│          │                         │   ↓ model detects turn   │
│          │   JSON: transcript      │ onTurn() → your LLM code │
│          │ ◄────────────────────── │   ↓ (sentence chunking)  │
│          │   binary: audio         │ TTS                      │
│ Speaker  │ ◄────────────────────── │                          │
└──────────┘                         └──────────────────────────┘

1. The client captures mic audio and sends it as binary WebSocket frames (16kHz mono 16-bit PCM).
2. Audio streams continuously to the transcriber session (created at start_call, lives for the entire call).
3. The STT model detects when the user finishes an utterance and fires onUtterance. All providers use model-driven turn detection — the client does not need to signal end-of-speech for STT.
4. Your onTurn() method runs — typically an LLM call.
5. The response is sentence-chunked and synthesized via TTS.
6. Audio streams back to the client for playback.

The client receives transcript_interim messages with partial results as the user speaks, so you can show real-time feedback in the UI.

Server API: withVoice

withVoice(Agent) adds the full voice pipeline to an Agent class.

Providers

Set providers as class properties. Class field initializers run after super(), so this.env is available.

import { withVoice, WorkersAIFluxSTT, WorkersAITTS } from "@cloudflare/voice";

const VoiceAgent = withVoice(Agent);

export class MyAgent extends VoiceAgent {
    transcriber = new WorkersAIFluxSTT(this.env.AI);
    tts = new WorkersAITTS(this.env.AI);
}

import { withVoice, WorkersAIFluxSTT, WorkersAITTS } from "@cloudflare/voice";

const VoiceAgent = withVoice(Agent);

export class MyAgent extends VoiceAgent<Env> {
    transcriber = new WorkersAIFluxSTT(this.env.AI);
    tts = new WorkersAITTS(this.env.AI);
}

For runtime model switching (for example, a Flux vs Nova 3 dropdown), override createTranscriber:

export class MyAgent extends VoiceAgent {
    tts = new WorkersAITTS(this.env.AI);

createTranscriber(connection) {
        return new WorkersAIFluxSTT(this.env.AI);
    }
}

export class MyAgent extends VoiceAgent<Env> {
    tts = new WorkersAITTS(this.env.AI);

createTranscriber(connection: Connection): Transcriber {
        return new WorkersAIFluxSTT(this.env.AI);
    }
}

onTurn(transcript, context)

Required. Called when the user finishes speaking and the transcript is ready. context.messages contains completed conversation history before this transcript. Append transcript exactly once when constructing an LLM message list.

Return a string, AsyncIterable<string>, or ReadableStream for streaming responses.

Simple response:

export class MyAgent extends VoiceAgent {
    transcriber = new WorkersAIFluxSTT(this.env.AI);
    tts = new WorkersAITTS(this.env.AI);

async onTurn(transcript, context) {
        return "You said: " + transcript;
    }
}

export class MyAgent extends VoiceAgent<Env> {
    transcriber = new WorkersAIFluxSTT(this.env.AI);
    tts = new WorkersAITTS(this.env.AI);

async onTurn(transcript: string, context: VoiceTurnContext) {
        return "You said: " + transcript;
    }
}

Streaming response (recommended for LLM):

import { streamText } from "ai";
import { createAI } from "agents/models/ai-sdk";

export class MyAgent extends VoiceAgent {
    transcriber = new WorkersAIFluxSTT(this.env.AI);
    tts = new WorkersAITTS(this.env.AI);

async onTurn(transcript, context) {
        const ai = createAI({ binding: this.env.AI });

const result = streamText({
            model: ai("@cf/moonshotai/kimi-k2.6"),
            system: "You are a helpful voice assistant. Keep responses concise.",
            messages: [
                ...context.messages.map((m) => ({
                    role: m.role,
                    content: m.content,
                })),
                { role: "user", content: transcript },
            ],
            abortSignal: context.signal,
        });

return result.textStream;
    }
}

import { streamText } from "ai";
import { createAI } from "agents/models/ai-sdk";

export class MyAgent extends VoiceAgent<Env> {
    transcriber = new WorkersAIFluxSTT(this.env.AI);
    tts = new WorkersAITTS(this.env.AI);

async onTurn(transcript: string, context: VoiceTurnContext) {
        const ai = createAI({ binding: this.env.AI });

const result = streamText({
            model: ai("@cf/moonshotai/kimi-k2.6"),
            system: "You are a helpful voice assistant. Keep responses concise.",
            messages: [
                ...context.messages.map((m) => ({
                    role: m.role as "user" | "assistant",
                    content: m.content,
                })),
                { role: "user", content: transcript },
            ],
            abortSignal: context.signal,
        });

return result.textStream;
    }
}

The context object provides:

Lifecycle hooks

Pipeline hooks

Intercept and transform data at each pipeline stage. Return null to skip the current utterance.

import {} from "agents";

export class MyAgent extends VoiceAgent {
    transcriber = new WorkersAIFluxSTT(this.env.AI);
    tts = new WorkersAITTS(this.env.AI);

afterTranscribe(transcript, connection) {
        if (transcript.length < 3) return null;
        return transcript;
    }

beforeSynthesize(text, connection) {
        return text.replace(/\bAI\b/g, "A.I.");
    }

async onTurn(transcript, context) {
        return transcript;
    }
}

import { type Connection } from "agents";

export class MyAgent extends VoiceAgent<Env> {
    transcriber = new WorkersAIFluxSTT(this.env.AI);
    tts = new WorkersAITTS(this.env.AI);

afterTranscribe(transcript: string, connection: Connection) {
        if (transcript.length < 3) return null;
        return transcript;
    }

beforeSynthesize(text: string, connection: Connection) {
        return text.replace(/\bAI\b/g, "A.I.");
    }

async onTurn(transcript: string, context: VoiceTurnContext) {
        return transcript;
    }
}

Convenience methods

Configuration options

Pass options to withVoice() as the second argument:

const VoiceAgent = withVoice(Agent, {
    historyLimit: 20,
    audioFormat: "mp3",
    maxMessageCount: 1000,
});

const VoiceAgent = withVoice(Agent, {
    historyLimit: 20,
    audioFormat: "mp3",
    maxMessageCount: 1000,
});

Server API: withVoiceInput

withVoiceInput(Agent) adds STT-only voice input — no TTS, no LLM, no response generation. Use this for dictation, search-by-voice, or any UI where you need speech-to-text without a conversational agent.

import { Agent } from "agents";
import { withVoiceInput, WorkersAINova3STT } from "@cloudflare/voice";

const InputAgent = withVoiceInput(Agent);

export class DictationAgent extends InputAgent {
    transcriber = new WorkersAINova3STT(this.env.AI);

onTranscript(text, connection) {
        console.log("User said:", text);
    }
}

import { Agent } from "agents";
import { withVoiceInput, WorkersAINova3STT } from "@cloudflare/voice";

const InputAgent = withVoiceInput(Agent);

export class DictationAgent extends InputAgent<Env> {
    transcriber = new WorkersAINova3STT(this.env.AI);

onTranscript(text: string, connection: Connection) {
        console.log("User said:", text);
    }
}

onTranscript(text, connection)

Called after each utterance is transcribed. Override this to process the transcript.

Hooks

withVoiceInput supports the same lifecycle hooks as withVoice:

• beforeCallStart(connection) — return false to reject
• onCallStart(connection), onCallEnd(connection), onInterrupt(connection)
• createTranscriber(connection) — override for runtime model switching
• afterTranscribe(transcript, connection) — filter or transform transcripts

It does not have TTS hooks (beforeSynthesize, afterSynthesize) or onTurn.

Client API: React hooks

useVoiceAgent

Wraps VoiceClient for withVoice agents. Manages connection, mic capture, playback, silence detection, and interrupt detection.

import { useVoiceAgent } from "@cloudflare/voice/react";

const selectedSpeakerId = "default";

const {
    status, // "idle" | "listening" | "thinking" | "speaking"
    transcript, // TranscriptMessage[] — conversation history
    interimTranscript, // string | null — real-time partial transcript
    metrics, // VoicePipelineMetrics | null
    audioLevel, // number (0–1) — current mic RMS level
    isMuted, // boolean
    connected, // boolean — WebSocket connected
    error, // string | null
    outputDeviceError, // string | null — non-fatal speaker routing issue
    startCall, // () => Promise<void>
    endCall, // () => void
    toggleMute, // () => void
    sendText, // (text: string) => void — bypass STT
    sendJSON, // (data: Record<string, unknown>) => void
    lastCustomMessage, // unknown — last non-voice message from server
} = useVoiceAgent({
    agent: "MyAgent",
    name: "default",
    host: window.location.host,
    outputDeviceId: selectedSpeakerId, // Optional audiooutput device ID
    enabled: true,
});

Use enabled: false when the app must wait for async connection prerequisites, such as a user-scoped capability token. While disabled, the hook does not create or connect a VoiceClient, returns the idle disconnected state, and action callbacks such as startCall(), sendText(), and sendJSON() are safe no-ops.

When enabled changes to true, the hook connects with the current options. The first enable is treated as an initial connection, so onReconnect only fires for later connection identity changes while the hook remains enabled.

Output device selection

Pass outputDeviceId to route assistant playback to a selected speaker when the browser supports HTMLMediaElement.setSinkId():

const [outputDeviceId, setOutputDeviceId] = useState("default");

const voice = useVoiceAgent({
    agent: "MyAgent",
    outputDeviceId,
});

const [outputDeviceId, setOutputDeviceId] = useState("default");

const voice = useVoiceAgent({
    agent: "MyAgent",
    outputDeviceId,
});

Use a MediaDeviceInfo.deviceId from navigator.mediaDevices.enumerateDevices() where kind === "audiooutput". "default" and undefined use the system default output. Browsers without sink selection support continue playing through the default output and set outputDeviceError when a non-default output is requested. Device labels may be blank until the user grants microphone permission, so refresh device lists after startCall() if you show a speaker picker.

Tuning options

Changing tuning options triggers a client reconnect (the connection key includes them).

useVoiceInput

Lightweight hook for dictation and voice-to-text. Accumulates user transcripts into a single string.

```tsx import { useVoiceInput } from "@cloudflare/voice/react";

function Dictation() { const { transcript, // string — accumulated text from all utterances interimTranscript, // string | null — current partial transcript isListening, // boolean audioLevel, // number (0–1) isMuted, // boolean error, // string | null start, // () => Promise<void> stop, // () => void toggleMute, // () => void clear, // () => void — clear accumulated transcript } = useVoiceInput({ agent: "DictationAgent" });