DeepgramAgent

Defined in: src/providers/agent/deepgram/DeepgramAgent.ts:73

Deepgram Agent API provider — covers STT + LLM + TTS in a single WebSocket.

See

• DeepgramAgentConfig for configuration options
• BaseAgentProvider for the shared agent provider base class
• DeepgramAgentEvent for the event types emitted via onDeepgramAgentEvent

Extends

• BaseAgentProvider

Constructors

Constructor

new DeepgramAgent(config, logger?): DeepgramAgent;

Defined in: src/providers/agent/deepgram/DeepgramAgent.ts:86

Parameters

Returns

DeepgramAgent

Overrides

BaseAgentProvider.constructor

Properties

Accessors

isProxyMode

Get Signature

get protected isProxyMode(): boolean;

Defined in: src/providers/base/BaseProvider.ts:286

Whether the provider is in proxy mode.

Returns

boolean

true when proxyUrl is set.

Inherited from

BaseAgentProvider.isProxyMode

Methods

assertAuth()

protected assertAuth(): void;

Defined in: src/providers/base/BaseProvider.ts:272

Validate that auth is configured (either apiKey or proxyUrl).

Returns

void

Remarks

Call this in onInitialize() for any provider that requires external authentication. Native providers (NativeSTT, NativeTTS) and in-browser providers (WebLLM) should NOT call this method.

Throws

ProviderInitializationError Thrown when neither apiKey nor proxyUrl is set.

Inherited from

BaseAgentProvider.assertAuth

assertReady()

protected assertReady(): void;

Defined in: src/providers/base/BaseProvider.ts:255

Guard that throws if the provider has not been initialized.

Returns

void

Remarks

Call at the start of any method that requires the provider to be ready.

Throws

Error Thrown with a descriptive message when initialized is false.

Inherited from

BaseAgentProvider.assertReady

cleanupPendingState()

protected cleanupPendingState(): void;

Defined in: src/providers/base/BaseAgentProvider.ts:611

Clean up all pending promises and timers. Call from your onDispose() or connection close handler.

Returns

void

Inherited from

BaseAgentProvider.cleanupPendingState

clearBufferedAssistantText()

protected clearBufferedAssistantText(): void;

Defined in: src/providers/base/BaseAgentProvider.ts:526

Discard assistant text buffered for a response that is being abandoned.

Returns

void

Remarks

Call this when the user starts a new utterance (barge-in, or the provider’s “user started speaking” event). Text left in the buffer by a response nobody consumed would otherwise be handed to the next turn’s iterator as if it were that turn’s answer.

Inherited from

BaseAgentProvider.clearBufferedAssistantText

configureInputFormat()

configureInputFormat(_metadata): void;

Defined in: src/providers/base/BaseAgentProvider.ts:236

Learn the format the input provider actually produces.

Parameters

Returns

void

Remarks

Called by configureSTTFromMetadata() once the input provider is known, which is the only point where the agent can find out that a caller-supplied microphone does not match preferredInputSampleRate. The base implementation is a no-op; subclasses override it to resample, renegotiate, or warn.

Inherited from

BaseAgentProvider.configureInputFormat

connect()

connect(): Promise<void>;

Defined in: src/providers/agent/deepgram/DeepgramAgent.ts:147

Open the WebSocket and complete the Deepgram Agent handshake.

Returns

Promise<void>

Remarks

The connection follows a 3-step handshake:

1. Open — a WebSocket is opened to wss://agent.deepgram.com/v1/agent/converse (or the configured proxy URL). The client waits without sending anything.
2. Welcome → Settings — the server sends a Welcome message containing a request_id. The client responds with a Settings message that configures listen (STT), think (LLM), and speak (TTS) providers.
3. SettingsApplied — the server acknowledges the settings. At this point the connection is ready and the keep-alive timer (8 s interval) is started.

The entire handshake must complete within the configured timeout (default 10 s, override via config.timeout). If the timeout elapses, the WebSocket is closed and the returned promise rejects.

Idempotent / piggyback behavior: if the WebSocket is already fully connected (settingsApplied === true), the call returns immediately. If a connection is in progress, the caller piggybacks on the existing handshake promise rather than opening a second socket.

Throws

If the handshake does not complete within the timeout window.

Throws

If the WebSocket emits an error or closes before the handshake finishes.

Overrides

BaseAgentProvider.connect

disconnect()

disconnect(): Promise<void>;

Defined in: src/providers/base/BaseAgentProvider.ts:246

No-op — the agent connection is persistent.

Returns

Promise<void>

Remarks

The orchestrator calls stt.disconnect() / tts.disconnect() during turn-taking and barge-in. For agent providers, the connection must not be torn down between turns. The real close happens in dispose().

Inherited from

BaseAgentProvider.disconnect

dispose()

dispose(): Promise<void>;

Defined in: src/providers/base/BaseProvider.ts:154

Clean up resources and dispose of the provider.

Returns

Promise<void>

Remarks

Delegates to the subclass hook onDispose and resets the initialized flag. If the provider is not initialized, the call is a no-op.

Throws

Re-throws any error raised by onDispose.

Inherited from

BaseAgentProvider.dispose

emitAssistantText()

protected emitAssistantText(text): void;

Defined in: src/providers/base/BaseAgentProvider.ts:507

Deliver the assistant’s response text. Resolves the pending generateFromMessages iterator.

Parameters

Returns

void

Remarks

When no iterator has registered yet — the orchestrator’s STT → LLM flow runs asynchronously after emitUserTranscription, so a fast agent can respond first — the text is buffered and consumed by the next iterator for this turn instead of being dropped.

Inherited from

BaseAgentProvider.emitAssistantText

emitAudioChunk()

protected emitAudioChunk(data): void;

Defined in: src/providers/base/BaseAgentProvider.ts:545

Forward a binary audio chunk to the output provider.

Parameters

Returns

void

Remarks

The audio format is re-asserted before every chunk. Barge-in tears down playback via BrowserAudioOutput.stop(), which clears the player’s metadata, and nothing else replays it — without this, raw PCM arriving after the first barge-in has no format to decode with and the rest of the session is silent. emitOutputMetadata is a cheap field assignment on the output side, so re-asserting per chunk costs nothing and removes the race between “playback stopped” and “next chunk”.

Inherited from

BaseAgentProvider.emitAudioChunk

emitOutputMetadata()

protected emitOutputMetadata(): void;

Defined in: src/providers/agent/deepgram/DeepgramAgent.ts:236

Emit audio format metadata so the output provider can decode PCM.

Returns

void

Nothing — metadata is delivered via the registered metadataCallback.

Remarks

Called after the handshake and again before every audio chunk (because BrowserAudioOutput.stop() clears metadata during barge-in). Must therefore be cheap and side-effect free — subclasses should do nothing but call this.metadataCallback?.(metadata).

Overrides

BaseAgentProvider.emitOutputMetadata

emitUserTranscription()

protected emitUserTranscription(text): void;

Defined in: src/providers/base/BaseAgentProvider.ts:473

Emit a user transcription. Creates a new per-turn audio promise and triggers the orchestrator’s STT -> LLM flow.

Parameters

Returns

void

Inherited from

BaseAgentProvider.emitUserTranscription

finalize()

finalize(): Promise<void>;

Defined in: src/providers/base/BaseAgentProvider.ts:425

Wait for the agent to finish sending all audio for the current turn.

Returns

Promise<void>

A promise that resolves when the current turn’s audio is complete or the 30 s timeout elapses.

Remarks

Resolves when the subclass calls markAudioDone, or after a 30-second safety timeout — whichever comes first. The timeout prevents the orchestrator from hanging indefinitely if the agent never signals completion.

Inherited from

BaseAgentProvider.finalize

generate()

generate(prompt, options?): Promise<AsyncIterable<string, any, any>>;

Defined in: src/providers/base/BaseAgentProvider.ts:317

Generate a response from a single user prompt.

Parameters

Returns

Promise<AsyncIterable<string, any, any>>

An async iterable that yields the assistant’s response as a single text chunk.

Remarks

Wraps the prompt in a user message and delegates to generateFromMessages.

Throws

AbortError if the supplied options.signal is aborted before the server responds.

Inherited from

BaseAgentProvider.generate

generateFromMessages()

generateFromMessages(_messages, _options?): Promise<AsyncIterable<string, any, any>>;

Defined in: src/providers/base/BaseAgentProvider.ts:349

Returns an async iterable that yields the assistant’s response text.

Parameters

Returns

Promise<AsyncIterable<string, any, any>>

An async iterable that yields the assistant’s response as a single text chunk.

Remarks

Agent APIs manage their own conversation context. This method does NOT re-send the message history — the server already has it. Instead, it returns an iterable that blocks until the server sends the assistant’s response text, then yields it as a single chunk.

The returned iterator resolves when the subclass calls emitAssistantText. If the connection closes before a response arrives, the iterator rejects via rejectPendingLLM.

Throws

AbortError if the supplied _options.signal is aborted before the server responds.

Inherited from

BaseAgentProvider.generateFromMessages

getConfig()

getConfig(): BaseProviderConfig;

Defined in: src/providers/base/BaseProvider.ts:187

Get a shallow copy of the current provider configuration.

Returns

BaseProviderConfig

A new object containing all current configuration values.

Inherited from

BaseAgentProvider.getConfig

initialize()

initialize(): Promise<void>;

Defined in: src/providers/base/BaseProvider.ts:127

Initialize the provider, making it ready for use.

Returns

Promise<void>

Remarks

Calls the subclass hook onInitialize. If the provider has already been initialized the call is a no-op.

Throws

ProviderInitializationError Thrown when onInitialize rejects. The original error is wrapped with the provider class name for diagnostics.

Inherited from

BaseAgentProvider.initialize

injectAgentMessage()

injectAgentMessage(message): void;

Defined in: src/providers/agent/deepgram/DeepgramAgent.ts:294

Force the agent to speak a specific message via TTS.

Parameters

Returns

void

Remarks

Sends an InjectAgentMessage frame. The server synthesizes the provided text through the configured TTS provider and streams the resulting audio back to the client. The message is also added to the conversation history as an assistant turn. This is useful for announcements or guided prompts that bypass the LLM entirely.

The agent may refuse the injection if it is currently speaking; in that case an injection_refused event is emitted via onDeepgramAgentEvent.

Example

agent.injectAgentMessage('Welcome! How can I help you today?');

injectUserMessage()

injectUserMessage(content): void;

Defined in: src/providers/agent/deepgram/DeepgramAgent.ts:270

Inject a text message as if the user spoke it.

Parameters

Returns

void

Remarks

Sends an InjectUserMessage frame to the Agent API. The agent treats this exactly like a transcribed user utterance — it will appear in the conversation history and trigger an LLM response. Use this when you want to drive the conversation programmatically without actual speech input (e.g. pre-filling a question from a button click).

Example

agent.injectUserMessage('What is the weather in San Francisco?');

isAudioReady()

isAudioReady(chunk): boolean;

Defined in: src/providers/base/BaseAgentProvider.ts:458

Check whether an audio chunk is ready for playback.

Parameters

Returns

boolean

true when the chunk contains a non-empty audio buffer.

Inherited from

BaseAgentProvider.isAudioReady

isFinal()

isFinal(result): boolean;

Defined in: src/providers/base/BaseAgentProvider.ts:294

Check whether a transcription is a final result.

Parameters

Returns

boolean

true when the result has isFinal set.

Inherited from

BaseAgentProvider.isFinal

isInterim()

isInterim(_result): boolean;

Defined in: src/providers/base/BaseAgentProvider.ts:284

Check whether a transcription is an interim (non-final) result.

Parameters

Returns

boolean

Always false for agent providers.

Remarks

Agent providers only surface final transcriptions — always returns false.

Inherited from

BaseAgentProvider.isInterim

isPreflight()

isPreflight(_result): boolean;

Defined in: src/providers/base/BaseAgentProvider.ts:271

Check whether a transcription is a speculative preflight result.

Parameters

Returns

boolean

Always false for agent providers.

Remarks

Agent providers do not emit preflight results — always returns false.

Inherited from

BaseAgentProvider.isPreflight

isReady()

isReady(): boolean;

Defined in: src/providers/base/BaseProvider.ts:178

Check whether the provider has been initialized and is ready.

Returns

boolean

true when initialize has completed successfully and dispose has not yet been called.

Inherited from

BaseAgentProvider.isReady

isUtteranceComplete()

isUtteranceComplete(result): boolean;

Defined in: src/providers/base/BaseAgentProvider.ts:258

Check whether a transcription marks the end of an utterance.

Parameters

Returns

boolean

true when the result has utteranceComplete set.

Inherited from

BaseAgentProvider.isUtteranceComplete

isWebSocketConnected()

isWebSocketConnected(): boolean;

Defined in: src/providers/agent/deepgram/DeepgramAgent.ts:415

Check whether the underlying WebSocket is currently open.

Returns

boolean

true if the WebSocket exists and its readyState is OPEN; false otherwise (including during the handshake, after close, or if connect has not yet been called).

markAudioDone()

protected markAudioDone(): void;

Defined in: src/providers/base/BaseAgentProvider.ts:558

Signal that all audio for the current turn has been sent. Resolves the pending finalize() call.

Returns

void

Inherited from

BaseAgentProvider.markAudioDone

onAudio()

onAudio(callback): void;

Defined in: src/providers/base/BaseAgentProvider.ts:438

Register the audio callback.

Parameters

Returns

void

Inherited from

BaseAgentProvider.onAudio

onConfigUpdate()

protected onConfigUpdate(_config): void;

Defined in: src/providers/base/BaseProvider.ts:242

Hook called after updateConfig merges new values.

Parameters

Returns

void

Remarks

The default implementation is a no-op. Override in subclasses to react to runtime configuration changes (e.g. reconnect with a new API key).

Inherited from

BaseAgentProvider.onConfigUpdate

onDeepgramAgentEvent()

onDeepgramAgentEvent(callback): void;

Defined in: src/providers/agent/deepgram/DeepgramAgent.ts:404

Register a callback for Deepgram Agent-specific events.

Parameters

Returns

void

Remarks

The callback receives a discriminated union (DeepgramAgentEvent) whose type field identifies the event. The full set of event types:

• user_started_speaking — the user began speaking (barge-in detected).
• agent_thinking — the LLM is generating a response; content contains the partial or complete “thinking” text.
• agent_started_speaking — TTS audio playback has begun; includes totalLatency, ttsLatency, and tttLatency timing metrics.
• agent_audio_done — the agent has finished sending audio for this turn.
• conversation_text — a finalized transcript for either the user or assistant role, with the full content string.
• function_call — the agent is requesting execution of one or more functions; each entry includes id, name, arguments, and clientSide flag.
• error — a server-side error with code and description.
• warning — a server-side warning with code and description.
• prompt_updated — confirms a updatePrompt change was applied.
• speak_updated — confirms a updateSpeak change was applied.
• think_updated — confirms an updateThink change was applied.
• injection_refused — a injectAgentMessage or injectUserMessage was refused; message contains the reason.

onDispose()

protected onDispose(): Promise<void>;

Defined in: src/providers/agent/deepgram/DeepgramAgent.ts:106

Provider-specific disposal logic.

Returns

Promise<void>

Remarks

Subclasses must implement this method to release any resources acquired during onInitialize (e.g. close connections, free memory).

Overrides

BaseAgentProvider.onDispose

onInitialize()

protected onInitialize(): Promise<void>;

Defined in: src/providers/agent/deepgram/DeepgramAgent.ts:102

Provider-specific initialization logic.

Returns

Promise<void>

Remarks

Subclasses must implement this method to perform any setup required before the provider can be used (e.g. validate credentials, open connections, load models).

Overrides

BaseAgentProvider.onInitialize

onMetadata()

onMetadata(callback): void;

Defined in: src/providers/base/BaseAgentProvider.ts:448

Register the metadata callback.

Parameters

Returns

void

Inherited from

BaseAgentProvider.onMetadata

onTranscription()

onTranscription(callback): void;

Defined in: src/providers/base/BaseAgentProvider.ts:203

Register the transcription callback.

Parameters

Returns

void

Inherited from

BaseAgentProvider.onTranscription

processAudio()

processAudio(chunk): void;

Defined in: src/providers/base/BaseAgentProvider.ts:193

Forward microphone audio to the agent connection.

Parameters

Returns

void

Remarks

Alias for sendAudio — called by the orchestrator’s input queue.

See

sendAudio

Inherited from

BaseAgentProvider.processAudio

processChunk()

processChunk(_text): void;

Defined in: src/providers/base/BaseAgentProvider.ts:411

No-op alias for sendText.

Parameters

Returns

void

Inherited from

BaseAgentProvider.processChunk

rejectPendingLLM()

protected rejectPendingLLM(error): void;

Defined in: src/providers/base/BaseAgentProvider.ts:571

Reject the pending LLM generator with an error.

Parameters

Returns

void

Inherited from

BaseAgentProvider.rejectPendingLLM

resolveApiKey()

protected resolveApiKey(): Promise<string>;

Defined in: src/providers/base/BaseProvider.ts:321

Resolve the API key, calling the factory if apiKey is a function.

Returns

Promise<string>

The resolved API key string, or 'proxy' in proxy mode.

Inherited from

BaseAgentProvider.resolveApiKey

resolveAuthHeader()

protected resolveAuthHeader(defaultAuthType?): Promise<string | undefined>;

Defined in: src/providers/base/BaseProvider.ts:366

Resolve Authorization header value for the configured auth type.

Parameters

Returns

Promise<string | undefined>

The Authorization header value, or undefined in proxy mode.

Remarks

If apiKey is a factory function it is called to get a fresh token. Returns the header value for REST or server-side WebSocket connections:

• 'token' → 'Token <apiKey>'
• 'bearer' → 'Bearer <apiKey>'

Returns undefined in proxy mode.

Inherited from

BaseAgentProvider.resolveAuthHeader

resolveBaseUrl()

protected resolveBaseUrl(defaultUrl?): string | undefined;

Defined in: src/providers/base/BaseProvider.ts:307

Resolve the base URL for this provider.

Parameters

Returns

string | undefined

The resolved URL, or undefined when all sources are unset.

Remarks

Priority: proxyUrl > endpoint > defaultUrl.

For WebSocket providers (this.type === 'websocket'), the proxy URL’s http(s) scheme is automatically converted to ws(s).

When no URL is configured and defaultUrl is undefined, the return value is undefined — this lets SDK-based providers (Anthropic, OpenAI) fall back to their own built-in defaults.

Inherited from

BaseAgentProvider.resolveBaseUrl

resolveWsProtocols()

protected resolveWsProtocols(defaultAuthType?): Promise<string[] | undefined>;

Defined in: src/providers/base/BaseProvider.ts:342

Resolve WebSocket subprotocol for authentication.

Parameters

Returns

Promise<string[] | undefined>

Subprotocol array for new WebSocket(url, protocols), or undefined.

Remarks

If apiKey is a factory function it is called to get a fresh token. Returns the subprotocol array for direct mode based on authType:

• 'token' → ['token', apiKey] (Deepgram default)
• 'bearer' → ['bearer', apiKey] (OAuth/Bearer tokens)

Returns undefined in proxy mode (no client-side auth needed).

Inherited from

BaseAgentProvider.resolveWsProtocols

sendAudio()

sendAudio(chunk): void;

Defined in: src/providers/agent/deepgram/DeepgramAgent.ts:230

Send a raw audio chunk to the Deepgram Agent for speech recognition.

Parameters

Returns

void

Remarks

This is a no-op when the WebSocket is not in the OPEN state — audio sent before connect completes or after the socket closes is silently dropped.

Overrides

BaseAgentProvider.sendAudio

sendKeepAlive()

sendKeepAlive(): void;

Defined in: src/providers/agent/deepgram/DeepgramAgent.ts:370

Send a keep-alive signal to prevent the WebSocket from timing out.

Returns

void

Remarks

Sends a KeepAlive JSON frame. You typically do not need to call this manually — an automatic keep-alive timer is started after the handshake completes, firing every 8 seconds. This method is exposed for advanced use cases where you need to reset the idle timer or send an extra heartbeat.

sendText()

sendText(_chunk): void;

Defined in: src/providers/base/BaseAgentProvider.ts:404

No-op — TTS synthesis is handled server-side.

Parameters

Returns

void

Inherited from

BaseAgentProvider.sendText

startKeepAlive()

protected startKeepAlive(intervalMs?, sendFn): void;

Defined in: src/providers/base/BaseAgentProvider.ts:594

Start an auto keep-alive timer. Call from your handshake completion.

Parameters

Returns

void

Example

this.startKeepAlive(8_000, () => {
  this.ws.send(JSON.stringify({ type: 'KeepAlive' }));
});

Inherited from

BaseAgentProvider.startKeepAlive

stopKeepAlive()

protected stopKeepAlive(): void;

Defined in: src/providers/base/BaseAgentProvider.ts:600

Stop the keep-alive timer.

Returns

void

Inherited from

BaseAgentProvider.stopKeepAlive

updateConfig()

updateConfig(config): void;

Defined in: src/providers/base/BaseProvider.ts:201

Merge partial configuration updates into the current config.

Parameters

Returns

void

Remarks

After merging, the subclass hook onConfigUpdate is called so providers can react to changed values at runtime.

Inherited from

BaseAgentProvider.updateConfig

updatePrompt()

updatePrompt(prompt): void;

Defined in: src/providers/agent/deepgram/DeepgramAgent.ts:310

Update the system prompt mid-session.

Parameters

Returns

void

Remarks

Sends an UpdatePrompt frame. The change takes effect on the next LLM inference — it does not retroactively alter previous turns. A prompt_updated event is emitted via onDeepgramAgentEvent when the server acknowledges the change. Use this to adapt the agent’s persona or instructions in response to user actions without restarting the session.

updateSpeak()

updateSpeak(speak): void;

Defined in: src/providers/agent/deepgram/DeepgramAgent.ts:333

Change the TTS provider or voice mid-session.

Parameters

Returns

void

Remarks

Sends an UpdateSpeak frame. The new voice is used starting with the next agent utterance. A speak_updated event is emitted via onDeepgramAgentEvent when the server acknowledges the change.

Example

// Switch to a different Deepgram Aura voice mid-conversation
agent.updateSpeak({
  provider: { type: 'deepgram', model: 'aura-2-orpheus-en' },
});

updateThink()

updateThink(think): void;

Defined in: src/providers/agent/deepgram/DeepgramAgent.ts:357

Change the LLM provider or model mid-session.

Parameters

Returns

void

Remarks

Sends an UpdateThink frame. The new model is used starting with the next inference turn. A think_updated event is emitted via onDeepgramAgentEvent when the server acknowledges the change.

Example

// Upgrade to a more capable model mid-conversation
agent.updateThink({
  provider: { type: 'open_ai', model: 'gpt-4o' },
  prompt: 'You are an expert technical assistant.',
});