Provider Matrix

CompositeVoice supports 25 provider companies across 36 provider classes (including 1 agent provider), plus 14 input/output providers for the 5-role pipeline. This page organizes them by company so you can see everything a single vendor offers.

Pipeline Role Matrix

Every provider and the pipeline role(s) it fills. Multi-role providers cover two or more adjacent stages — use them for simpler configs (3 providers instead of 5). Agent providers cover stt + llm + tts in a single connection (1 provider instead of 5).

The pipeline requires all 5 roles to be filled. If both input and stt are uncovered, NullInput is auto-filled (text-only, no microphone). If both tts and output are uncovered, NullOutput is auto-filled (text-only, no speakers). If an STT is provided without an input, MicrophoneInput is auto-filled. If a TTS is provided without an output, BrowserAudioOutput is auto-filled. If no llm is provided, AnthropicLLM (claude-haiku-4-5) is auto-filled. Agent providers like DeepgramAgent cover stt + llm + tts — the SDK auto-fills MicrophoneInput and BrowserAudioOutput for the remaining input and output roles.

Audio Input / Output (Pipeline I/O)

These providers handle the input and output roles in the 5-role pipeline. They are not tied to any vendor.

MicrophoneInput buffers audio frames in the input queue while the STT WebSocket connects, then flushes them in order — no audio is ever lost. BufferInput does the same for programmatic audio sources. NullInput covers both input and stt roles for text-only pipelines — no microphone is requested.

BrowserAudioOutput handles AudioContext resumption and buffers frames in the output queue during speaker setup. NullOutput covers both tts and output roles for text-only pipelines — no audio is played.

Multi-role providers like NativeSTT (input+stt), NativeTTS (tts+output), NullInput (input+stt), and NullOutput (tts+output) cover multiple pipeline roles. When using them, you do not need separate input/output providers.

Deepgram

DeepgramSTT (V1/Nova) features: Interim results, smart formatting, auto-punctuation, speaker diarization, entity detection, keyword boosting, profanity filter, redaction (PCI/SSN), numerals conversion, VAD events, word-level timestamps, configurable endpointing, utterance buffering, multichannel transcription. Models: nova-3 (recommended), nova-3-medical, nova-2 (+ domain variants), nova (legacy).

DeepgramFlux (V2/Flux) features: Turn-based conversation model, eager end-of-turn detection (configurable thresholds: eot_threshold 0.5–0.9, eager_eot_threshold 0.3–0.9), end-of-turn timeout (eot_timeout_ms), keyterm support, word confidence scores. Events: StartOfTurn, EagerEndOfTurn, TurnResumed, EndOfTurn, Update. Models: flux-general-en. Only provider that supports the eager LLM pipeline.

TTS features: Real-time streaming synthesis, linear16/mulaw/alaw encoding, configurable sample rate (8–48 kHz), word-level timing metadata.

TTS models: Aura 2 (recommended — 40 English voices + 10 Spanish voices), Aura 1 (legacy — 12 English voices).

Guides: DeepgramSTT · DeepgramFlux · DeepgramTTS · Examples: 20, 21, 22, 23, 24

Deepgram Agent

DeepgramAgent connects to the Deepgram Voice Agent API (wss://agent.deepgram.com/v1/agent/converse) via a single WebSocket that collapses the entire STT + LLM + TTS pipeline into one connection. Deepgram handles speech recognition, LLM inference, and text-to-speech synthesis server-side — the client only sends raw audio and receives raw audio back.

Agent features: Configurable listen (STT), think (LLM), and speak (TTS) sub-providers via the Settings message, greeting message on session start, mid-session prompt/voice/model updates (updatePrompt, updateSpeak, updateThink), message injection (injectUserMessage, injectAgentMessage), client-side and server-side function calling, conversation context pre-seeding, latency metrics (AgentStartedSpeaking with total_latency, tts_latency, ttt_latency), keep-alive, barge-in support.

Think (LLM) providers: OpenAI, Anthropic, Google, Groq, AWS Bedrock.

Speak (TTS) providers: Deepgram, ElevenLabs, Cartesia, OpenAI, AWS Polly.

Examples: 70

Anthropic

LLM features: Streaming via SSE, system prompts extracted to top-level system parameter (Anthropic API convention), maxTokens required (default 1024), AbortSignal cancellation for the eager pipeline, temperature and topP controls.

Models: claude-haiku-4-5 (fastest), claude-sonnet-4-6 (balanced), claude-opus-4-6 (most capable).

Guides: AnthropicLLM · Examples: 00, 30, 31

OpenAI

STT features: Realtime API transcription intent (wss://api.openai.com/v1/realtime?intent=transcription), interim results via transcript deltas, server VAD or semantic VAD turn detection for automatic turn-taking, input noise reduction (near-field/far-field), prompt-based vocabulary steering, language hint, ephemeral client-secret auth via async apiKey factories (WebSocket subprotocols), manual finalize() commits, automatic reconnection with session re-configuration.

STT models: gpt-4o-mini-transcribe (default), gpt-4o-transcribe, whisper-1, gpt-realtime-whisper (native streaming, manual commits only).

LLM features: GPT model family, streaming token generation, organizationId for multi-org accounts, temperature/topP/maxTokens controls.

LLM models: gpt-4o-mini, gpt-4o, gpt-4-turbo, gpt-3.5-turbo.

TTS features: 6 voices (alloy, echo, fable, onyx, nova, shimmer), quality/speed tradeoff via model selection (tts-1 fast, tts-1-hd quality), 5 output formats (mp3, opus, aac, flac, wav), speed control (0.25–4.0x), 4096 character limit per request, endpoint for Azure OpenAI compatibility.

Guides: OpenAIRealtimeSTT · OpenAILLM · OpenAITTS · Examples: 40, 41, 42

Groq

LLM features: Ultra-fast LPU-based inference (lowest latency of any cloud LLM), OpenAI-compatible API, groqApiKey convenience alias, wide range of open-source models.

Models: llama-3.3-70b-versatile, mixtral-8x7b-32768, gemma2-9b-it, llama-3.1-8b-instant.

Guides: GroqLLM · Examples: 60

Google Gemini

LLM features: OpenAI-compatible endpoint, geminiApiKey convenience alias, auto-configured base URL (generativelanguage.googleapis.com/v1beta/openai).

Models: gemini-2.0-flash, gemini-1.5-pro, gemini-1.5-flash.

Guides: GeminiLLM · Examples: 100

Mistral

LLM features: Strong multilingual support, OpenAI-compatible API, mistralApiKey convenience alias.

Models: mistral-small-latest, mistral-medium-latest, mistral-large-latest.

Guides: MistralLLM · Examples: 110

AssemblyAI

STT features: Interim results, word boosting for domain vocabulary, word-level timestamps and confidence, automatic reconnection with exponential backoff, base64-encoded audio, graceful terminate_session on disconnect, configurable sample rate.

Guides: AssemblyAISTT · Examples: 70

Soniox

STT features: Interim results via provisional/confirmed token streaming, 60+ languages with automatic detection and hints, endpoint detection with a <end> token for turn-taking, speaker diarization, per-token language identification, domain context for specialized vocabulary, manual finalize(), binary audio frames (no base64 overhead), temporary API key support via async apiKey factories, automatic reconnection with exponential backoff.

Guides: SonioxSTT

Gladia

STT features: Interim results (partial transcripts), configurable server-side endpointing for turn-taking, language pinning and auto-detection with per-utterance code switching, word-level timestamps and confidence, binary audio frames (no base64 overhead), session-token WebSocket URLs (only the init POST needs credentials), processing region selection (us-west / eu-west), automatic reconnection resuming the same session, graceful stop_recording on disconnect.

Guides: GladiaSTT

Speechmatics

STT features: Interim results via partial transcripts, 50+ languages with output locale and domain packs, end-of-utterance detection for turn-taking, configurable accuracy/latency (operatingPoint, maxDelay), speaker diarization, custom vocabulary (additionalVocab), manual forceEndOfUtterance(), binary audio frames (no base64 overhead), temporary key (JWT) support via async apiKey factories, automatic reconnection with exponential backoff.

Guides: SpeechmaticsSTT

Rev AI

STT features: Interim results via partial hypotheses, punctuated and capitalized finals with per-word timestamps and confidence, 9 languages (en, fr, de, it, ja, ko, cmn, pt, es), profanity filtering, disfluency removal, custom vocabularies, speaker-switch labels (machine_v2 transcriber), configurable segment duration for faster finals, raw/FLAC/WAV audio via content_type, graceful EOS end-of-stream, automatic reconnection with exponential backoff, async apiKey factories for short-lived tokens.

Guides: RevAISTT

ElevenLabs

STT features: Scribe V2 Realtime (~150ms latency), 90+ languages with auto-detection, VAD and manual commit strategies, interim results (partial transcripts), word-level timestamps and confidence, base64-encoded audio, three auth methods (API key, proxy, single-use token), BCP 47 / ISO 639-1 / ISO 639-3 language code auto-mapping, configurable VAD sensitivity, previousText context, zero-retention mode.

TTS features: Voice cloning controls (stability 0–1, similarityBoost 0–1), BOS/EOS stream-input protocol, word-level alignment, 6 output formats (pcm_16000, pcm_22050, pcm_24000, pcm_44100, mp3_44100_128, ulaw_8000), multilingual models.

TTS models: eleven_turbo_v2_5 (fast), eleven_turbo_v2, eleven_multilingual_v2, eleven_monolingual_v1.

Guides: ElevenLabsSTT · ElevenLabsTTS · Examples: 80, 81

Cartesia

TTS features: Ultra-low-latency streaming, context-based streaming (context_id + continue flag preserves prosody across chunks), emotion controls (emotion_name:intensity tags), speed multiplier, 4 PCM encodings (s16le, f32le, mulaw, alaw), word-level timestamps, configurable sample rate.

Models: sonic-2 (latest, lowest latency), sonic, sonic-multilingual.

Guides: CartesiaTTS · Examples: 90

Speechify

TTS features: Catalog and instant-cloned voices via voiceId, English and multilingual Simba models, 4 output formats (mp3, wav, ogg, aac), optional loudness and text normalization, emotion/pitch/speed via SSML <prosody> tags in the input.

Models: simba-3.2 (latest), simba-3.0, simba-multilingual, simba-english (default).

Guides: SpeechifyTTS

Murf AI

TTS features: Gen2 model with natural, studio-quality voices, per-voice speaking styles (style), rate and pitch controls (-50 to 50), prosody variation (0–5), multilingual voices via locale, 5 output formats (mp3, wav, flac, alaw, ulaw), configurable sample rate (8–48 kHz) and channel type, base64 inline audio (no second download request).

Models: GEN2 (default).

Guides: MurfTTS

LMNT

TTS features: Catalog and instant-cloned voices via voice, 31 languages with auto-detection, 7 output formats (mp3, wav, aac, ulaw, webm, pcm_s16le, pcm_f32le), configurable sample rate (8/16/24 kHz), expressiveness (temperature) and stability (topP) controls.

Models: blizzard (Blizzard 2.0, default).

Guides: LMNTTTS

Smallest.ai

TTS features: Ultra-low-latency Waves Lightning models, catalog and cloned voices via voiceId, 12 languages (English, Hindi, Spanish, and 9 Indian languages), 5 output formats (wav, mp3, pcm, ulaw, alaw), configurable sample rate (8–44.1 kHz) and speed (0.5–2.0x).

Models: lightning_v3.1 (default), lightning_v3.1_pro (curated voice pool, improved naturalness).

Guides: SmallestTTS

Rime

TTS features: Per-model voice catalogs via speaker, flagship coda and expressive arcana model families plus the low-latency mist family, 6 output formats selected via the Accept header (mp3, wav, ogg, webm, pcm, mulaw), configurable sampling rate, speed (speedAlpha) and normalization (noTextNormalization) controls on mistv2, multilingual synthesis on Coda and Arcana.

Models: coda (flagship), arcana (default), arcanav3, arcanav2, mistv3 (fastest), mistv2.

Guides: RimeTTS

MiniMax

TTS features: 300+ system voices across 30+ languages plus cloned voices via voiceId, emotion/speed/volume/pitch controls, 4 output formats (mp3, wav, flac, pcm), languageBoost pronunciation hints, custom pronunciation dictionary, optional groupId for older group-scoped keys.

Models: speech-2.8-hd / speech-2.8-turbo (latest), speech-2.6-hd / speech-2.6-turbo, speech-02-hd (default) / speech-02-turbo, speech-01-hd / speech-01-turbo.

Guides: MiniMaxTTS

Fish Audio

TTS features: Catalog voices via referenceId, instant voice cloning via inline binary reference audio, model generation selected with the model HTTP header, 4 output formats (mp3, wav, pcm, opus), prosody controls (speed 0.5–2.0, volume), latency modes (normal / balanced ~300ms time-to-first-audio), text normalization, configurable mp3 bitrate and chunk length.

Models: s2.1-pro (recommended for production), s2.1-pro-free (free tier, default), s2-pro, s1.

Peer dependency: Requests are MessagePack-encoded (Content-Type: application/msgpack), so FishAudioTTS requires the optional peer dependency @msgpack/msgpack — install it with pnpm add @msgpack/msgpack. It is loaded lazily during initialize().

Guides: FishAudioTTS

Google Cloud

GoogleSTT (batch) features: Synchronous speech:recognize REST transcription of complete recordings (up to 60 seconds / 10 MB per request), one final utteranceComplete result per utterance, automatic punctuation, profanity filtering, phrase hints (keywords → speechContexts), up to 3 alternative language candidates, word-level time offsets in metadata, WAV/FLAC header auto-detection. Models: latest_short (best for voice-agent turns), latest_long, telephony, telephony_short, medical_dictation, medical_conversation, and legacy models. No streaming variant: Google’s StreamingRecognize is gRPC-only (v1 and v2) with no public WebSocket endpoint, so a zero-dependency live provider is not possible.

TTS features: Full Google voice catalog — Chirp 3: HD (latest generation), Neural2, Studio, WaveNet, Polyglot, News, Casual, Standard (Journey voices were retired into Chirp 3: HD) — SSML input (auto-detected via a leading <speak tag), speaking rate / pitch / volume gain controls, configurable sample rate, device effects profiles, 5 encodings (MP3, OGG_OPUS, LINEAR16, MULAW, ALAW).

Auth: Google Cloud API key injected as the X-goog-api-key header (direct mode) or server-side by the proxy (googleCloudApiKey config registers both the google-tts and google-stt routes). OAuth2 service accounts are out of scope. This is separate from the Gemini LLM key/route above.

Guides: GoogleSTT · GoogleTTS

Microsoft Azure

STT features: Real-time recognition over the Speech service WebSocket protocol (the same wire format as the official Speech SDK), 100+ locales, interim hypotheses, service-side end-of-utterance detection driving utteranceComplete, continuous recognition across turns, simple/detailed (NBest + confidence) output, profanity handling, browser-safe query-parameter auth (subscription key or 10-minute bearer token), automatic reconnection with exponential backoff.

TTS features: Hundreds of neural voices across 140+ locales, SSML synthesis with automatic XML escaping, speaking styles via <mstts:express-as> (with styleDegree), rate/pitch via <prosody>, output formats spanning mp3, wav (riff), ogg/webm opus, and raw pcm via X-Microsoft-OutputFormat.

Auth: Both providers accept a Speech resource key (Ocp-Apim-Subscription-Key) or an async apiKey factory returning a 10-minute STS bearer token; both share the azureSpeechApiKey + azureSpeechRegion proxy configuration.

Guides: AzureSTT · AzureTTS

Amazon Web Services

Auth: AWS requests are authenticated with Signature Version 4 rather than an API key — the SDK signs Polly requests (Authorization header) and presigns Transcribe WebSocket URLs (X-Amz-* query parameters) using a built-in WebCrypto signer; no AWS SDK dependency. In browsers, use temporary STS/Cognito credentials via an async credentials factory, or route through the proxy (aws proxy config) so credentials stay server-side.

TranscribeSTT features: Interim (IsPartial) and final results with word-level timing and confidence, partial-results stabilization (high/medium/low), custom vocabularies and vocabulary filters (remove/mask/tag), speaker partitioning, automatic language identification with candidate languageOptions, session IDs, binary application/vnd.amazon.eventstream audio framing with CRC32 validation.

PollyTTS features: Neural, generative, long-form, and standard engines, 100+ voices across 40+ language variants, SSML input (textType: 'ssml'), pronunciation lexicons, 4 output formats (mp3, ogg_vorbis, ogg_opus, pcm), configurable sample rate.

Guides: TranscribeSTT · PollyTTS

Browser Built-ins

STT features: Zero dependencies, works offline, 50+ languages via browser, continuous mode, interim results, maxAlternatives, startTimeout, managed audio (browser controls the microphone directly).

TTS features: Zero dependencies, works offline, voice enumeration via getAvailableVoices(), voice selection by name/language, rate/pitch/volume controls, pause/resume/cancel playback, runtime voice switching with setVoice(), managed audio (browser plays directly).

Limitations: NativeSTT requires Chromium (no Firefox). Both use managed audio — the SDK cannot access raw audio streams. No preflight signals. Best for prototyping.

Guides: NativeSTT · NativeTTS · Examples: 00

WebLLM (MLC AI)

LLM features: Fully offline after initial model download, all data stays in the browser, onLoadProgress callback for download UI, chatOpts for engine tuning, engine.interruptGenerate() abort support, no API keys needed.

Example models: Llama-3.2-1B-Instruct-q4f16_1-MLC (~500 MB), Phi-2-q4f16_1-MLC (~1.5 GB).

Guides: WebLLMLLM · Examples: 50

Feature comparison at a glance