Providers

CompositeVoice uses five pipeline roles — input (audio capture), STT (speech-to-text), LLM (large language model), TTS (text-to-speech), and output (audio playback). Mix and match any combination to build your voice pipeline. Some providers cover multiple roles (e.g., NativeSTT handles both input and stt).

Audio Input

MicrophoneInput

Captures audio from the browser’s microphone via getUserMedia and AudioContext. Use this when pairing with a WebSocket-based STT provider like DeepgramSTT or AssemblyAISTT.

import { MicrophoneInput } from 'composite-voice';

const input = new MicrophoneInput({
  sampleRate: 16000,        // audio sample rate in Hz
});

• Buffers audio frames in the input queue during STT connection — no audio is ever lost
• Works in all modern browsers that support getUserMedia
• Requires HTTPS or localhost

BufferInput

Accepts audio data pushed programmatically. Use this for server-side pipelines (Node.js, Bun, Deno) where there is no microphone.

import { BufferInput } from 'composite-voice';

const input = new BufferInput({
  sampleRate: 16000,
  encoding: 'linear16',
  channels: 1,
  bitDepth: 16,
});

// Push audio from any source (file, stream, WebSocket, etc.)
input.push(audioBuffer);

• Zero browser dependencies — no navigator, window, or AudioContext
• Works in Node.js, Bun, and Deno

WebRTCInput

Extracts mono linear16 PCM from a WebRTC MediaStreamTrack the application obtained from any peer connection — LiveKit, Daily, or a raw RTCPeerConnection. The app owns the connection and the track; the provider only runs the local processing graph.

import { WebRTCInput } from 'composite-voice';

const input = new WebRTCInput({ targetSampleRate: 16000 });

pc.ontrack = (event) => {
  if (event.track.kind === 'audio') input.setTrack(event.track);
};

• Swap sources live with setTrack() / setStream() (e.g. on active-speaker change)
• Never calls track.stop() — the app keeps ownership
• See the WebRTC guide for LiveKit and Daily examples

Speech-to-Text (STT)

NativeSTT

Uses the browser’s built-in Web Speech API. Zero API keys required. Best for prototyping and demos.

import { NativeSTT } from 'composite-voice';

const stt = new NativeSTT({
  language: 'en-US',        // BCP 47 language tag
  continuous: true,          // keep listening after each result
  interimResults: true,      // emit partial transcripts
  maxAlternatives: 1,        // number of recognition alternatives
});

• No API key needed
• Works offline
• Supports 50+ languages via the browser
• Managed audio — the browser controls the microphone directly
• Does not work in de-Googled browsers (Ungoogled Chromium, Brave) — the Web Speech API requires Google’s speech servers

API reference

DeepgramSTT

Production-grade real-time speech recognition via WebSocket using Deepgram’s V1 (Nova) API. Best accuracy across the widest range of languages.

import { DeepgramSTT } from 'composite-voice';

const stt = new DeepgramSTT({
  proxyUrl: '/api/proxy/deepgram',   // server proxy (recommended)
  // OR: apiKey: 'dg-...',           // direct API key (dev only)
  language: 'en',
  interimResults: true,
  options: {
    model: 'nova-3',          // nova-3 (recommended), nova-2, nova-3-medical
    smartFormat: true,         // auto-punctuation and formatting
    punctuation: true,
    profanityFilter: false,
    diarize: false,            // speaker identification
    endpointing: 300,          // ms of silence before end-of-speech
    utteranceEndMs: 1000,      // ms before utterance boundary
  },
});

• nova-3 (highest accuracy, recommended default), nova-2 (wider language support)
• Word-level confidence and timestamps
• Smart formatting and auto-punctuation
• Profanity filtering
• Speaker diarization
• VAD events

Does not support preflight/eager end-of-turn signals. For the eager LLM pipeline, use DeepgramFlux.

API reference

DeepgramFlux

Low-latency real-time speech recognition via WebSocket using Deepgram’s V2 (Flux) API. Supports eager end-of-turn signals for the eager LLM pipeline.

import { DeepgramFlux } from 'composite-voice';

const stt = new DeepgramFlux({
  proxyUrl: '/api/proxy/deepgram',   // server proxy (recommended)
  // OR: apiKey: 'dg-...',           // direct API key (dev only)
  options: {
    model: 'flux-general-en',
    eagerEotThreshold: 0.5,    // enables eager end-of-turn signals
    eotThreshold: 0.7,
  },
});

• Turn-based transcription via TurnInfo events
• Eager end-of-turn signals (EagerEndOfTurn → isPreflight: true)
• Configurable end-of-turn confidence thresholds
• Keyterm boosting for domain vocabulary
• Only STT provider that supports the eager LLM pipeline

API reference

AssemblyAISTT

Real-time speech recognition via WebSocket with word boosting for domain-specific vocabulary.

import { AssemblyAISTT } from 'composite-voice';

const stt = new AssemblyAISTT({
  proxyUrl: '/api/proxy/assemblyai',
  // OR: apiKey: '...',
  sampleRate: 16000,
  language: 'en',
  wordBoost: ['CompositeVoice', 'WebSocket'],  // boost domain terms
});

• Word boosting for domain vocabulary
• Word-level timestamps and confidence
• Automatic reconnection

API reference

ElevenLabsSTT

Real-time speech recognition via WebSocket using ElevenLabs Scribe V2 with ~150ms latency and 90+ language support.

import { ElevenLabsSTT } from 'composite-voice';

const stt = new ElevenLabsSTT({
  proxyUrl: '/api/proxy/elevenlabs',
  // OR: apiKey: '...',
  // OR: token: '...',             // single-use token
  model: 'scribe_v2_realtime',
  audioFormat: 'pcm_16000',
  language: 'en',                  // BCP 47, ISO 639-1, or ISO 639-3
  commitStrategy: 'vad',           // 'vad' (default) or 'manual'
  includeTimestamps: true,         // word-level timestamps
});

• VAD and manual commit strategies
• 90+ languages with auto-detection
• Word-level timestamps and confidence
• Three auth methods (API key, proxy, single-use token)
• Shares proxy config with ElevenLabsTTS

API reference

SonioxSTT

Real-time multilingual speech recognition via WebSocket with built-in endpoint detection for turn-taking.

import { SonioxSTT } from 'composite-voice';

const stt = new SonioxSTT({
  proxyUrl: '/api/proxy/soniox',
  // OR: apiKey: '...',              // direct or async temporary-key factory
  model: 'stt-rt-v5',
  audioFormat: 'pcm_s16le',
  sampleRate: 16000,
  languageHints: ['en', 'es'],       // bias recognition
  enableEndpointDetection: true,      // default — drives turn-taking
  enableSpeakerDiarization: false,
});

• 60+ languages with automatic detection
• Endpoint detection finalizes utterances when the speaker stops
• Speaker diarization and per-token language identification
• Domain context for specialized vocabulary
• Temporary API key support via async apiKey factories

API reference

GladiaSTT

Real-time speech recognition via Gladia’s v2 live API (Solaria models) with configurable server-side endpointing for turn-taking.

import { GladiaSTT } from 'composite-voice';

const stt = new GladiaSTT({
  proxyUrl: '/api/proxy/gladia',
  // OR: apiKey: '...',              // direct API key (dev only)
  model: 'solaria-1',
  encoding: 'wav/pcm',
  sampleRate: 16000,
  languages: ['en'],                 // pin or restrict language detection
  endpointing: 0.3,                  // seconds of silence before finalizing
  codeSwitching: false,              // re-detect language per utterance
});

• HTTP session init (POST /v2/live) + direct WebSocket streaming
• Server-side endpointing finalizes utterances when the speaker stops
• Language pinning and per-utterance code switching
• Word-level timestamps and confidence on final results
• Session token embedded in the WebSocket URL — reconnects resume the session

API reference

SpeechmaticsSTT

Real-time speech recognition via WebSocket with built-in end-of-utterance detection for turn-taking.

import { SpeechmaticsSTT } from 'composite-voice';

const stt = new SpeechmaticsSTT({
  proxyUrl: '/api/proxy/speechmatics',
  // OR: apiKey: async () => '...',  // temporary-key (JWT) factory for direct mode
  language: 'en',
  audioFormat: 'pcm_s16le',
  sampleRate: 16000,
  operatingPoint: 'enhanced',         // accuracy/latency trade-off
  endOfUtteranceSilenceTrigger: 0.75, // default — drives turn-taking
  enableSpeakerDiarization: false,
});

• 50+ languages with configurable output locale and domain packs
• End-of-utterance detection finalizes utterances when the speaker stops
• Speaker diarization and custom vocabulary (additionalVocab)
• Manual forceEndOfUtterance() for custom turn-taking
• Temporary key (JWT) support via async apiKey factories

API reference

RevAISTT

Real-time speech recognition via WebSocket with punctuated, confidence-scored final transcripts.

import { RevAISTT } from 'composite-voice';

const stt = new RevAISTT({
  proxyUrl: '/api/proxy/revai',
  // OR: apiKey: '...',              // direct or async token factory
  sampleRate: 16000,                 // raw audio: 8000-48000 Hz
  language: 'en',                    // en, fr, de, it, ja, ko, cmn, pt, es
  filterProfanity: false,            // English only
  removeDisfluencies: false,         // English only
  maxSegmentDurationSeconds: 10,     // force finals every 5-30 s
});

• Punctuated, capitalized finals with per-word timestamps and confidence
• 9 languages (en, fr, de, it, ja, ko, cmn, pt, es)
• Profanity filtering, disfluency removal, and custom vocabularies
• Speaker-switch labels with the machine_v2 transcriber
• Auth via access_token query parameter (injected server-side in proxy mode)

API reference

OpenAIRealtimeSTT

Real-time speech recognition via OpenAI’s Realtime API transcription intent, with server or semantic VAD turn detection.

import { OpenAIRealtimeSTT } from 'composite-voice';

const stt = new OpenAIRealtimeSTT({
  proxyUrl: '/api/proxy/openai-realtime',
  // OR: apiKey: '...',              // direct key or async ephemeral-secret factory
  model: 'gpt-4o-mini-transcribe',   // gpt-4o-transcribe, whisper-1, gpt-realtime-whisper
  language: 'en',                    // ISO 639-1 hint
  turnDetection: { type: 'server_vad' },  // default — drives turn-taking
  noiseReduction: 'near_field',      // optional input noise reduction
});

• Server VAD (volume-based) or semantic VAD (model-based) turn detection
• Optional input noise reduction (near-field / far-field)
• Prompt-based vocabulary steering for domain terms
• Ephemeral client-secret support via async apiKey factories (browser-safe auth over WebSocket subprotocols)
• 24 kHz mono PCM input (pair with MicrophoneInput({ sampleRate: 24000 }))

API reference

GoogleSTT

Batch (per-utterance) speech recognition via Google Cloud Speech-to-Text’s synchronous REST endpoint. Each transcribe(blob) call uploads a complete recording (up to 60 seconds) and emits one final result with utteranceComplete: true.

import { GoogleSTT } from 'composite-voice';

const stt = new GoogleSTT({
  proxyUrl: '/api/proxy/google-stt',
  // OR: apiKey: 'AIza...',            // Google Cloud API key (X-goog-api-key)
  language: 'en-US',
  encoding: 'WEBM_OPUS',               // matches MediaRecorder output
  sampleRate: 48000,
  model: 'latest_short',               // latest_short, latest_long, telephony, ...
  enableWordTimeOffsets: true,         // word timings in metadata.words
  keywords: ['CompositeVoice'],        // phrase hints via speechContexts
});

• Batch REST transcription — one final result per complete recording, no interim results
• 60 seconds / 10 MB of audio per request (synchronous speech:recognize limit)
• Automatic punctuation, profanity filtering, phrase hints, alternative languages
• Word-level time offsets in result metadata

Google’s streaming recognition (StreamingRecognize) is gRPC-only in both v1 and v2 — there is no public WebSocket endpoint, so no live variant exists. For real-time streaming STT use DeepgramSTT, AssemblyAISTT, or SonioxSTT.

API reference

AzureSTT

Microsoft Azure Speech real-time recognition via WebSocket, speaking the same wire protocol as the official Speech SDK.

import { AzureSTT } from 'composite-voice';

const stt = new AzureSTT({
  proxyUrl: '/api/proxy/azure-stt',
  // OR: apiKey: '...', region: 'eastus',  // string key or async token factory
  language: 'en-US',
  recognitionMode: 'conversation',   // conversation, interactive, dictation
  outputFormat: 'simple',            // 'detailed' adds NBest + confidence
});

• 100+ recognition locales
• Interim hypotheses plus final phrases with utteranceComplete turn-taking
• Continuous recognition across turns on one connection
• Query-parameter auth for browsers (subscription key or 10-minute bearer token)
• Zero dependencies — no microsoft-cognitiveservices-speech-sdk required

API reference

TranscribeSTT

Amazon Transcribe streaming speech recognition over WebSocket, authenticated with SigV4-presigned URLs. No AWS SDK required.

import { TranscribeSTT } from 'composite-voice';

const stt = new TranscribeSTT({
  proxyUrl: '/api/proxy/transcribe',
  // OR: credentials: async () => fetchTempCredentials(), region: 'us-east-1',
  languageCode: 'en-US',
  mediaEncoding: 'pcm',               // pcm, ogg-opus, flac
  sampleRate: 16000,
  enablePartialResultsStabilization: true,
  partialResultsStability: 'high',    // high, medium, low
});

• Partial (interim) and final results with word-level timing
• Partial-results stabilization for lower interim latency
• Custom vocabularies, vocabulary filters, and speaker partitioning
• Automatic language identification (identifyLanguage + languageOptions)
• Temporary-credentials support via async credentials factories (STS/Cognito)
• Built-in WebCrypto SigV4 presigning and event-stream framing

API reference

Large Language Models (LLM)

AnthropicLLM

Claude models via the Anthropic API. Uses a dedicated SDK (not OpenAI-compatible).

import { AnthropicLLM } from 'composite-voice';

const llm = new AnthropicLLM({
  proxyUrl: '/api/proxy/anthropic',
  model: 'claude-haiku-4-5',    // claude-haiku-4-5, claude-sonnet-4-6, claude-opus-4-6
  maxTokens: 1024,               // required (default: 1024)
});

• System prompts at top level (Anthropic API convention)
• Streaming via SSE
• AbortSignal cancellation for the eager pipeline

API reference

OpenAILLM

GPT models via the OpenAI API.

import { OpenAILLM } from 'composite-voice';

const llm = new OpenAILLM({
  proxyUrl: '/api/proxy/openai',
  model: 'gpt-4o-mini',
  // organizationId: 'org-...',  // for multi-org accounts
});

API reference

GroqLLM

Ultra-fast inference on Groq’s LPU hardware. Supports open-source models.

import { GroqLLM } from 'composite-voice';

const llm = new GroqLLM({
  proxyUrl: '/api/proxy/groq',
  model: 'llama-3.3-70b-versatile',  // or mixtral-8x7b-32768, gemma2-9b-it
});

• Lowest latency of any cloud LLM provider
• Wide range of open-source models

API reference

MistralLLM

Mistral models with strong multilingual support.

import { MistralLLM } from 'composite-voice';

const llm = new MistralLLM({
  proxyUrl: '/api/proxy/mistral',
  model: 'mistral-small-latest',  // or mistral-medium-latest, mistral-large-latest
});

API reference

GeminiLLM

Google Gemini models via their OpenAI-compatible endpoint.

import { GeminiLLM } from 'composite-voice';

const llm = new GeminiLLM({
  proxyUrl: '/api/proxy/gemini',
  model: 'gemini-2.0-flash',  // or gemini-1.5-pro, gemini-1.5-flash
});

API reference

WebLLMLLM

Run LLMs entirely in the browser via WebGPU. No API keys, no network, full privacy.

import { WebLLMLLM } from 'composite-voice';

const llm = new WebLLMLLM({
  model: 'Llama-3.2-1B-Instruct-q4f16_1-MLC',
  onLoadProgress: (progress) => {
    console.log(`Loading: ${(progress.progress * 100).toFixed(0)}%`);
  },
});

• All data stays in the browser
• Works offline after initial model download
• Requires a WebGPU-capable browser
• First load downloads model weights (100+ MB)

API reference

OpenAICompatibleLLM

Base class for any service that speaks the OpenAI chat completions format. Use this to connect custom or self-hosted models.

import { OpenAICompatibleLLM } from 'composite-voice';

const llm = new OpenAICompatibleLLM({
  endpoint: 'https://my-model-server.example.com/v1',
  model: 'my-custom-model',
  apiKey: '...',
});

API reference

Text-to-Speech (TTS)

NativeTTS

Uses the browser’s built-in SpeechSynthesis API. Zero API keys required.

import { NativeTTS } from 'composite-voice';

const tts = new NativeTTS({
  voiceName: 'Samantha',    // partial match against available voices
  voiceLang: 'en-US',       // BCP 47 fallback filter
  rate: 1.0,                // speech rate
  pitch: 0,                 // semitones (-20 to 20)
});

• No API key needed
• Works offline
• Managed audio — the browser plays directly
• Supports pause, resume, and cancel
• Voice enumeration via getAvailableVoices()

API reference

DeepgramTTS

Low-latency real-time streaming TTS via WebSocket with Aura 2 voices.

import { DeepgramTTS } from 'composite-voice';

const tts = new DeepgramTTS({
  proxyUrl: '/api/proxy/deepgram',
  voice: 'aura-2-thalia-en',    // thalia, andromeda, janus, proteus, orion, luna, arcas
  sampleRate: 24000,
  outputFormat: 'linear16',
});

• Lowest latency streaming TTS
• Word-level timing metadata
• Aura 2 voice models

API reference

OpenAITTS

OpenAI text-to-speech via REST. Returns complete audio in one request.

import { OpenAITTS } from 'composite-voice';

const tts = new OpenAITTS({
  proxyUrl: '/api/proxy/openai',
  model: 'tts-1',          // tts-1 (fast) or tts-1-hd (quality)
  voice: 'nova',           // alloy, echo, fable, onyx, nova, shimmer
  responseFormat: 'mp3',   // mp3, opus, aac, flac, wav
  speed: 1.0,              // 0.25 to 4.0
});

• Six distinct voices
• Quality/speed tradeoff via model selection
• 4096 character limit per request

API reference

ElevenLabsTTS

High-quality voice cloning and synthesis via WebSocket streaming.

import { ElevenLabsTTS } from 'composite-voice';

const tts = new ElevenLabsTTS({
  proxyUrl: '/api/proxy/elevenlabs',
  voiceId: 'your-voice-id',           // from ElevenLabs dashboard
  modelId: 'eleven_turbo_v2_5',       // turbo_v2_5, turbo_v2, multilingual_v2
  stability: 0.5,                      // voice consistency (0-1)
  similarityBoost: 0.75,              // voice fidelity (0-1)
  outputFormat: 'pcm_16000',          // pcm_16000, pcm_22050, pcm_24000, mp3_44100_128
});

• Voice cloning
• Multilingual models
• Stability and similarity controls
• Multiple output formats

API reference

CartesiaTTS

Ultra-low-latency streaming TTS with emotion controls.

import { CartesiaTTS } from 'composite-voice';

const tts = new CartesiaTTS({
  proxyUrl: '/api/proxy/cartesia',
  voiceId: 'your-voice-id',
  modelId: 'sonic-2',           // sonic-2 (latest), sonic, sonic-multilingual
  language: 'en',
  outputEncoding: 'pcm_s16le',
  outputSampleRate: 16000,
  speed: 'normal',              // or 'slow', 'fast'
  emotion: ['positivity:high'], // emotion tags
});

• Context-based streaming links chunks into coherent utterances
• Emotion controls
• Word-level timestamps
• sonic-2 model delivers the lowest latency

API reference

SpeechifyTTS

Speechify Simba text-to-speech via REST. Returns complete audio in one request.

import { SpeechifyTTS } from 'composite-voice';

const tts = new SpeechifyTTS({
  proxyUrl: '/api/proxy/speechify',
  voiceId: 'geffen_32',        // from GET /v1/voices or a cloned voice
  model: 'simba-3.2',          // simba-english, simba-multilingual, simba-3.0, simba-3.2
  audioFormat: 'mp3',          // mp3, wav, ogg, aac
  language: 'en-US',           // optional; auto-detected when omitted
});

• Catalog voices and instant voice cloning
• English and multilingual Simba models
• Emotion, pitch, and speed via SSML <prosody> tags in the input

API reference

MurfTTS

Murf AI Gen2 text-to-speech via REST. Returns complete audio in one request.

import { MurfTTS } from 'composite-voice';

const tts = new MurfTTS({
  proxyUrl: '/api/proxy/murf',
  voiceId: 'en-US-natalie',    // from GET /v1/speech/voices
  format: 'mp3',               // mp3, wav, flac, alaw, ulaw
  style: 'Conversational',     // per-voice speaking styles
  rate: 0,                     // -50 to 50
  pitch: 0,                    // -50 to 50
  variation: 1,                // 0 to 5 — prosody variation
});

• Gen2 model with natural, studio-quality voices
• Per-voice speaking styles (Conversational, Promo, …)
• Rate, pitch, and prosody variation controls
• Multilingual voices via the locale option

API reference

LMNTTTS

LMNT Blizzard text-to-speech via REST. Returns complete audio in one request.

import { LMNTTTS } from 'composite-voice';

const tts = new LMNTTTS({
  proxyUrl: '/api/proxy/lmnt',
  voice: 'leah',           // from GET /v1/ai/voice/list or a cloned voice
  model: 'blizzard',       // LMNT's current speech model
  format: 'mp3',           // mp3, wav, aac, ulaw, webm, pcm_s16le, pcm_f32le
  language: 'en',          // optional; auto-detected when omitted
  temperature: 0.7,        // expressiveness (lower = more neutral)
  topP: 0.9,               // stability (lower = more consistent)
});

• Catalog voices and instant voice cloning
• 31 languages via the Blizzard model
• Expressiveness (temperature) and stability (topP) controls

API reference

SmallestTTS

Smallest.ai Lightning text-to-speech via the Waves REST API. Returns complete audio in one request.

import { SmallestTTS } from 'composite-voice';

const tts = new SmallestTTS({
  proxyUrl: '/api/proxy/smallest',
  voiceId: 'meher',            // Waves catalog voice or a cloned voice
  model: 'lightning_v3.1',     // lightning_v3.1, lightning_v3.1_pro
  outputFormat: 'wav',         // wav, mp3, pcm, ulaw, alaw
  sampleRate: 24000,           // 8000, 16000, 24000, 44100
  speed: 1.0,                  // 0.5 to 2.0
});

• Ultra-low-latency Lightning v3.1 and v3.1 Pro models
• 12 languages (English, Hindi, Spanish, and 9 Indian languages) plus voice cloning
• Telephony-friendly ulaw/alaw output at 8 kHz

API reference

RimeTTS

Rime text-to-speech via REST. Returns complete audio in one request.

import { RimeTTS } from 'composite-voice';

const tts = new RimeTTS({
  proxyUrl: '/api/proxy/rime',
  speaker: 'astra',            // from Rime's per-model voice catalogs
  model: 'arcana',             // coda, arcana, arcanav3, arcanav2, mistv3, mistv2
  audioFormat: 'mp3',          // mp3, wav, ogg, webm, pcm, mulaw
  language: 'en',              // ISO 639-1 or 639-2/3 code
});

• Flagship coda, expressive arcana, and low-latency mist model families
• Output format selected via the Accept header (raw audio bytes)
• Speed and normalization controls (speedAlpha, noTextNormalization) on mistv2

API reference

MiniMaxTTS

MiniMax Speech text-to-speech via REST. Returns complete audio in one request.

import { MiniMaxTTS } from 'composite-voice';

const tts = new MiniMaxTTS({
  proxyUrl: '/api/proxy/minimax',
  voiceId: 'English_expressive_narrator', // system voice or a cloned voice
  model: 'speech-02-hd',       // speech-2.8/2.6/02/01, each in -hd and -turbo
  audioFormat: 'mp3',          // mp3, wav, flac, pcm
  emotion: 'calm',             // optional emotion control
});

• 300+ system voices across 30+ languages, plus voice cloning
• Emotion, speed, volume, and pitch controls via config options
• Optional groupId for older group-scoped API keys (sent as ?GroupId=)

API reference

FishAudioTTS

Fish Audio speech models (S1, S2 Pro, S2.1 Pro) via REST with msgpack-encoded requests. Returns complete audio in one request.

Requires the optional peer dependency @msgpack/msgpack (>=3.0.0) — Fish Audio’s API takes MessagePack request bodies, which also carry binary reference audio for instant voice cloning. Install it with pnpm add @msgpack/msgpack. This is the only TTS provider with a peer dependency.

import { FishAudioTTS } from 'composite-voice';

const tts = new FishAudioTTS({
  proxyUrl: '/api/proxy/fishaudio',
  referenceId: 'your-voice-id', // voice model from the Fish Audio catalog
  model: 's2.1-pro',            // s1, s2-pro, s2.1-pro, s2.1-pro-free (HTTP header)
  format: 'mp3',                // mp3, wav, pcm, opus
  latency: 'balanced',          // 'normal' (stable) or 'balanced' (~300ms TTFA)
});

• Catalog voices via referenceId, instant voice cloning via inline references
• Model generation selected with the model HTTP header
• Prosody controls (speed, volume) and text normalization
• Msgpack wire format (Content-Type: application/msgpack); requires @msgpack/msgpack

API reference

GoogleTTS

Google Cloud Text-to-Speech via REST. Returns complete audio in one request.

import { GoogleTTS } from 'composite-voice';

const tts = new GoogleTTS({
  proxyUrl: '/api/proxy/google-tts',
  languageCode: 'en-US',                  // BCP-47 language/region
  voiceName: 'en-US-Chirp3-HD-Kore',      // Chirp 3: HD, Neural2, Studio, WaveNet, ...
  audioEncoding: 'MP3',                   // MP3, OGG_OPUS, LINEAR16, MULAW, ALAW
  speakingRate: 1.0,                      // 0.25 – 4.0
  pitch: 0,                               // semitones, -20 to +20
});

• Full Google voice catalog: Chirp 3: HD (latest), Neural2, Studio, WaveNet, Polyglot, News, Casual, Standard
• SSML input supported (text starting with <speak is sent as SSML)
• Rate, pitch, volume gain, sample rate, and device effects profiles
• API-key auth via the X-goog-api-key header (service accounts out of scope)

API reference

AzureTTS

Microsoft Azure Speech text-to-speech via REST (SSML). Returns complete audio in one request.

import { AzureTTS } from 'composite-voice';

const tts = new AzureTTS({
  proxyUrl: '/api/proxy/azure-tts',
  // OR: apiKey: '...', region: 'eastus',  // string key or async token factory
  voiceName: 'en-US-AriaNeural',           // required
  outputFormat: 'audio-24khz-48kbitrate-mono-mp3',
  style: 'cheerful',                       // optional speaking style
  rate: 1.1,                               // optional prosody rate multiplier
});

• Hundreds of neural voices across 140+ locales
• Speaking styles via <mstts:express-as>, rate/pitch via <prosody>
• User text is XML-escaped automatically before SSML embedding
• mp3, wav (riff), ogg/webm opus, and raw pcm output formats

API reference

PollyTTS

Amazon Polly text-to-speech via SigV4-signed REST calls. Returns complete audio in one request. No AWS SDK required.

import { PollyTTS } from 'composite-voice';

const tts = new PollyTTS({
  proxyUrl: '/api/proxy/polly',
  // OR: credentials: async () => fetchTempCredentials(), region: 'us-east-1',
  voiceId: 'Joanna',           // see Polly's DescribeVoices
  engine: 'neural',            // neural, generative, long-form, standard
  outputFormat: 'mp3',         // mp3, ogg_vorbis, ogg_opus, pcm
});

• Neural, generative, long-form, and standard engines
• SSML input via textType: 'ssml' for prosody and pronunciation control
• Pronunciation lexicons via lexiconNames
• Temporary-credentials support via async credentials factories (STS/Cognito)

API reference

Agent Providers

Agent providers collapse the STT + LLM + TTS pipeline into a single persistent connection. Instead of configuring three separate providers, you configure one agent provider that covers all three roles. The SDK auto-fills MicrophoneInput and BrowserAudioOutput for the remaining input and output roles.

DeepgramAgent

Connects to the Deepgram Voice Agent API via a single WebSocket. Deepgram handles speech recognition, LLM inference, and text-to-speech synthesis server-side — the client only sends raw audio and receives raw audio back.

import { CompositeVoice, DeepgramAgent } from 'composite-voice';

const voice = new CompositeVoice({
  providers: [
    new DeepgramAgent({
      proxyUrl: '/api/proxy/deepgram-agent',
      think: {
        provider: { type: 'open_ai', model: 'gpt-4o-mini' },
        prompt: 'You are a helpful voice assistant.',
      },
      speak: {
        provider: { type: 'deepgram', model: 'aura-2-thalia-en' },
      },
      greeting: 'Hello! How can I help you?',
    }),
  ],
});

• Covers stt + llm + tts — only 1 provider needed (SDK auto-fills MicrophoneInput + BrowserAudioOutput)
• Configurable LLM: OpenAI, Anthropic, Google, Groq, AWS Bedrock
• Configurable TTS: Deepgram, ElevenLabs, Cartesia, OpenAI, AWS Polly
• Mid-session updates: updatePrompt(), updateSpeak(), updateThink()
• Message injection: injectUserMessage(), injectAgentMessage()
• Client-side and server-side function calling via onFunctionCall callback
• Greeting message on session start
• Barge-in support
• Latency metrics via AgentStartedSpeaking events

Audio Output

BrowserAudioOutput

Plays audio through the browser’s AudioContext and speakers. Use this when pairing with a WebSocket-based or REST-based TTS provider like DeepgramTTS, ElevenLabsTTS, or OpenAITTS.

import { BrowserAudioOutput } from 'composite-voice';

const output = new BrowserAudioOutput();

• Handles AudioContext resumption after user gestures
• Buffers audio frames in the output queue during setup — no audio is ever lost

NullOutput

Silently discards all audio. Use this for server-side pipelines where there are no speakers.

import { NullOutput } from 'composite-voice';

const output = new NullOutput();

• Zero browser dependencies — no navigator, window, or AudioContext
• Works in Node.js, Bun, and Deno

WebRTCOutput

Renders TTS audio into a publishable MediaStreamTrack so the agent’s voice can be added to any WebRTC room.

import { WebRTCOutput } from 'composite-voice';

const output = new WebRTCOutput();
await output.initialize();
pc.addTrack(output.getTrack(), output.getStream());

• Accepts linear16 directly, mulaw/alaw via built-in G.711 decoding, mp3/opus via decodeAudioData
• stop() implements barge-in (clears scheduled audio immediately)
• See the WebRTC guide for full pipeline examples

Platform Inputs & Outputs

Platform providers connect the pipeline to call and chat platforms.

TwilioMediaStream

Duplex provider for Twilio Media Streams. Your server accepts Twilio’s <Connect><Stream> WebSocket and hands each socket to the provider.

import { TwilioMediaStream } from 'composite-voice';

const twilio = new TwilioMediaStream();
wss.on('connection', (socket) => twilio.attach(socket));

• Caller audio: mu-law 8 kHz mono, auto-configures STT via getMetadata()
• TTS audio back: mu-law passthrough or linear16 auto-converted (any rate)
• flush() resolves on Twilio’s mark echo; stop() sends clear (barge-in)
• Extras: onDtmf(), onCallEnded(), getCallSid(), getCustomParameters()
• See the Twilio guide for TwiML + server setup

VonageAudioSocket

Duplex provider for the Vonage Voice API WebSocket endpoint. Your NCCO connect action points at your server; each accepted socket is handed to the provider.

import { VonageAudioSocket } from 'composite-voice';

const vonage = new VonageAudioSocket();
wss.on('connection', (socket) => vonage.attach(socket));

• Caller audio: linear16 at the NCCO-negotiated rate (8/16/24 kHz)
• TTS audio back: linear16 passthrough/resample or G.711 decode, paced in 20 ms binary frames
• No mark protocol — flush() resolves when the pacing pump drains
• Extras: onDtmf(), getContentType()
• See the Vonage guide for NCCO + server setup

DiscordVoice

Puts the agent in a Discord voice channel. Your bot joins the channel with @discordjs/voice’s joinVoiceChannel and hands the connection to the provider.

import { DiscordVoice } from 'composite-voice';
import { joinVoiceChannel } from '@discordjs/voice';

const discord = new DiscordVoice();
discord.attach(joinVoiceChannel({ channelId, guildId, adapterCreator, selfDeaf: false }));

• Peer dependencies: @discordjs/voice, prism-media, plus an Opus codec (@discordjs/opus or opusscript)
• Speakers’ Opus packets are decoded to 48 kHz PCM and downmixed to mono for STT
• TTS audio (linear16, any rate) is resampled to 48 kHz stereo and played via an AudioPlayer
• Requires the GuildVoiceStates gateway intent and selfDeaf: false
• See the Discord guide for full bot setup

ZoomRtmsInput

Streams live Zoom meeting audio into the pipeline via Realtime Media Streams. Zero dependencies — the RTMS signaling/media WebSocket protocol (HMAC-SHA256 handshakes, keep-alives, base64 L16 audio) is implemented directly.

import { ZoomRtmsInput } from 'composite-voice';

const zoom = new ZoomRtmsInput({ clientId, clientSecret });

// From your meeting.rtms_started webhook:
await zoom.connect({ meetingUuid, rtmsStreamId, serverUrl });

• Receive-only by design — pair with NullOutput (you cannot inject audio into a Zoom meeting via RTMS)
• Mixed stream by default, or per-participant audio via dataOpt: 'per-participant' + onSpeakerAudio()
• 16 kHz linear16 mono by default (8/16/32/48 kHz configurable)
• Server-side only (the HMAC signature requires your client secret)
• See the Zoom RTMS guide for webhook wiring and marketplace prerequisites

GoogleMeetInput

Joins a Google Meet conference through the Meet Media API and mixes the conference audio into the pipeline. Zero dependencies — the WebRTC offer/answer exchange (spaces.connectActiveConference), session-control and media-stats data channels are implemented directly, following Google’s reference client.

import { GoogleMeetInput } from 'composite-voice';

const meet = new GoogleMeetInput({
  apiKey: async () => getOAuthAccessToken(), // meetings.conference.media.audio.readonly
  spaceName: 'spaces/abc-defg-hij',
});

• Developer Preview: the Cloud project, OAuth principal, and all participants must be enrolled
• Receive-only by design — pair with NullOutput
• Browser-only (RTCPeerConnection + Web Audio); emits 16 kHz linear16 mono
• Session status surfaced via onSessionStatus() (waiting / joined / disconnected)
• See the Google Meet guide for OAuth scopes and space resolution

TeamsCall

Joins a Microsoft Teams meeting as an external participant via Azure Communication Services Teams interop, with raw media access in both directions.

import { TeamsCall } from 'composite-voice';

const teams = new TeamsCall({
  token: acsUserAccessToken,
  meetingLink: 'https://teams.microsoft.com/l/meetup-join/...',
  displayName: 'Voice Agent',
});

• Peer dependencies: @azure/communication-calling (>= 1.13.1), @azure/communication-common
• Meeting audio (mixed) flows to STT as 16 kHz linear16; TTS is played into the meeting via a custom outgoing audio stream
• Accepts linear16 (any rate), mulaw/alaw, and mp3 TTS output
• Lobby-aware: onCallStateChanged() surfaces InLobby/Connected/Disconnected
• See the Teams guide for ACS resource + token setup

Choosing providers

For prototyping: NativeSTT + any LLM + NativeTTS — no API keys except the LLM.

For production: DeepgramSTT + AnthropicLLM + DeepgramTTS — best accuracy, lowest latency, streaming throughout.

For privacy: NativeSTT + WebLLMLLM + NativeTTS — everything runs in the browser. No data leaves the device.

For lowest latency: DeepgramFlux + GroqLLM + DeepgramTTS — eager end-of-turn signals, fastest LLM inference, low-latency streaming TTS.

For simplest config: DeepgramAgent — one provider replaces the entire STT + LLM + TTS pipeline. Deepgram handles everything server-side.