AttachableInputProvider<TTarget>
An audio input provider that connects to its platform through a per-call handle passed to attach().
Defined in: src/core/types/providers.ts:1748
An audio input provider that connects to its platform through a per-call handle passed to attach().
Remarks
Platform input providers often cannot capture audio until the application hands them a live, platform-specific object — the WebSocket Twilio opened to your server, a Discord VoiceConnection, the session parameters from a Zoom meeting.rtms_started webhook, or a remote WebRTC track. Providers declare that handle’s type by implementing attach().
When a provider in the providers array implements this interface, CompositeVoice.startListening() accepts the attach target as an optional, fully typed parameter and forwards it to attach() before capture starts:
const voice = new CompositeVoice({
providers: [twilio, new DeepgramSTT({ ... }), new AnthropicLLM({ ... })],
});
await voice.initialize();
wss.on('connection', (socket) => {
void voice.startListening(socket); // typed as TwilioStreamSocket
});
Calling attach() directly and then startListening() with no argument remains equally valid — the parameter is a convenience, not a requirement.
See
- InputAttachTarget for how the target type is inferred
- StartListeningArgs for the resulting
startListening()signature
Extends
Type Parameters
| Type Parameter | Default type | Description |
|---|---|---|
TTarget | unknown | The platform-specific handle type (e.g. TwilioStreamSocket, DiscordVoiceConnection). |
Properties
| Property | Modifier | Type | Default value | Description | Inherited from | Defined in |
|---|---|---|---|---|---|---|
roles | readonly | readonly ProviderRole[] | [] | Pipeline roles this provider covers. Remarks Each provider declares which stages of the 5-role audio pipeline it can fulfil. Single-role providers list one role (e.g. ['stt']); multi-role providers list every role they handle (e.g. ['input', 'stt'] for NativeSTT, which manages its own microphone access). The provider resolution algorithm reads this property to assign providers to pipeline slots. Base provider classes set sensible defaults: - BaseSTTProvider: ['stt'] - BaseLLMProvider: ['llm'] - BaseTTSProvider: ['tts'] See - ProviderRole for the possible role values - ResolvedPipeline for the resolved pipeline slots | AudioInputProvider.roles | src/core/types/providers.ts:206 |
type | readonly | ProviderType | undefined | The communication type this provider uses. See ProviderType | AudioInputProvider.type | src/core/types/providers.ts:184 |
Methods
attach()
attach(target): void | Promise<void>;
Defined in: src/core/types/providers.ts:1755
Attach the per-call platform handle this provider captures audio from.
Parameters
| Parameter | Type | Description |
|---|---|---|
target | TTarget | The platform-specific object (socket, connection, session, track, …) to bind to. |
Returns
void | Promise<void>
dispose()
dispose(): Promise<void>;
Defined in: src/core/types/providers.ts:226
Clean up resources and dispose of the provider.
Returns
Promise<void>
Remarks
Called by CompositeVoice during agent shutdown. The provider should close any open connections, clear buffers, and release resources.
Inherited from
getMetadata()
getMetadata(): AudioMetadata;
Defined in: src/core/types/providers.ts:1533
Get the audio format metadata for the captured audio.
Returns
AudioMetadata describing the audio format
Remarks
Returns metadata describing the format of audio chunks this provider will emit. Used by the pipeline to auto-configure the STT provider’s encoding, sample rate, and channel settings.
See
AudioMetadata for the metadata type
Inherited from
AudioInputProvider.getMetadata
initialize()
initialize(): Promise<void>;
Defined in: src/core/types/providers.ts:217
Initialize the provider and allocate any required resources.
Returns
Promise<void>
Remarks
Called by CompositeVoice during agent startup. The provider should be ready to process requests after this method resolves.
Throws
Error if initialization fails (e.g., invalid API key, network error)
Inherited from
isActive()
isActive(): boolean;
Defined in: src/core/types/providers.ts:1508
Check whether the input source is actively capturing audio.
Returns
boolean
true when audio is being captured (between start() and stop(), and not paused)
Inherited from
isReady()
isReady(): boolean;
Defined in: src/core/types/providers.ts:233
Check if the provider is initialized and ready to process requests.
Returns
boolean
true if the provider has been initialized and is operational
Inherited from
onAudio()
onAudio(callback): void;
Defined in: src/core/types/providers.ts:1519
Register a callback to receive captured audio chunks.
Parameters
| Parameter | Type | Description |
|---|---|---|
callback | (chunk) => void | Function invoked with each AudioChunk |
Returns
void
Remarks
The provider calls this callback with each chunk of captured audio. Must be called before start.
Inherited from
pause()
pause(): void;
Defined in: src/core/types/providers.ts:1493
Temporarily pause audio capture without releasing the source.
Returns
void
Remarks
Used by the turn-taking system to mute capture during TTS playback. Resume with resume.
Inherited from
resume()
resume(): void;
Defined in: src/core/types/providers.ts:1500
Resume audio capture after a pause.
Returns
void
See
Inherited from
start()
start(): void;
Defined in: src/core/types/providers.ts:1453
Start capturing audio from the input source.
Returns
void
Remarks
After calling start(), the provider begins delivering audio chunks via the callback registered with onAudio. Must be called after initialize.
Inherited from
stop()
stop(): void;
Defined in: src/core/types/providers.ts:1468
Stop capturing audio and release the input source.
Returns
void
Remarks
Stops audio delivery. The provider can be restarted with start.
Duplex providers cover both 'input' and 'output', so a bare stop() is ambiguous — implement stopCapture and stopPlayback to say which side is meant.
Inherited from
stopCapture()?
optional stopCapture(): void;
Defined in: src/core/types/providers.ts:1484
Stop capturing audio, explicitly, without touching playback.
Returns
void
Remarks
The pipeline calls this instead of stop whenever it means “stop listening” — so a provider that fills both roles never has to infer intent from its own state. Inferring it is unsafe: a barge-in raised while the agent is still thinking has no audio playing to distinguish it from a stop-listening, and guessing wrong leaves the agent deaf for the rest of the session.
Optional. Providers that only cover 'input' can omit it — stop() is already unambiguous for them.