TwilioMediaStream

Defined in: src/providers/io/twilio/TwilioMediaStream.ts:242

Duplex input/output provider for Twilio Media Streams phone calls.

Remarks

TwilioMediaStream covers both the 'input' and 'output' pipeline roles for a single phone call:

• Input — Twilio media messages (base64 mu-law, 8 kHz mono) are decoded and emitted as AudioChunk objects while the provider is started and not paused. getMetadata() reports mulaw/8000/mono so the pipeline can auto-configure STT.
• Output — TTS audio is converted to base64 mu-law and sent to Twilio as media messages. mu-law @ 8 kHz passes through untouched; linear16 at any sample rate is resampled to 8 kHz and G.711-encoded. Any other format makes configure() throw with instructions for fixing the TTS configuration.
• Flush — flush() sends a mark message; Twilio echoes the mark back once playback reaches it, which resolves the flush promise.
• Barge-in — stop() sends a clear message so Twilio drops its buffered audio immediately, and settles any pending flushes as cancelled.

The application owns the WebSocket server: accept each Twilio connection and pass the socket to attach(). Calling attach() again replaces (and unwires) the previous socket.

Duplex method notes: stop() serves both roles — it halts input emission and clears Twilio’s playback buffer. pause()/resume() apply to the input side only (the turn-taking controller uses them to gate capture during agent speech); Twilio buffers outbound audio server-side and offers no way to pause playback mid-call.

Data-flow diagram:

Twilio "media"(base64 mu-law) ──▶ decode ──▶ onAudio(chunk) ──▶ [STT]
[TTS] ──▶ enqueue(chunk) ──▶ mu-law/8k? passthrough : resample+encode ──▶ "media" ──▶ Twilio
flush() ──▶ "mark" ──▶ Twilio echo ──▶ resolve
stop()  ──▶ "clear" ──▶ Twilio drops buffer (barge-in)

Example

import { TwilioMediaStream } from 'composite-voice';

const twilio = new TwilioMediaStream({ debug: true });
await twilio.initialize();

twilio.onDtmf((digit) => console.log('Caller pressed', digit));
twilio.onCallEnded(() => console.log('Call ended'));

wss.on('connection', (socket) => twilio.attach(socket));

See

• TwilioMediaStreamConfig for configuration options
• TwilioStreamSocket for the accepted socket shape

Implements

• AudioInputProvider
• AudioOutputProvider

Constructors

Constructor

new TwilioMediaStream(config?): TwilioMediaStream;

Defined in: src/providers/io/twilio/TwilioMediaStream.ts:352

Creates a new TwilioMediaStream instance.

Parameters

Returns

TwilioMediaStream

Remarks

No credentials are needed — the WebSocket handed to attach() is already authenticated by your server (Twilio connects to the URL you put in your TwiML).

Example

const twilio = new TwilioMediaStream({ debug: true });

Properties

Methods

attach()

attach(socket): void;

Defined in: src/providers/io/twilio/TwilioMediaStream.ts:437

Attach an accepted Twilio WebSocket to the provider.

Parameters

Returns

void

Remarks

Call this from your WebSocket server’s connection handler. The socket is duck-typed (TwilioStreamSocket): both Node ws sockets (on('message', ...)) and browser-style sockets (addEventListener('message', ...)) work. If a socket is already attached, it is detached first (its listeners are removed and any pending flushes are settled).

Example

wss.on('connection', (socket) => {
  twilio.attach(socket);
});

See

detach() to unwire manually

configure()

configure(metadata): void;

Defined in: src/providers/io/twilio/TwilioMediaStream.ts:805

Configure the output with the format of incoming TTS audio.

Parameters

Returns

void

Remarks

Twilio only accepts G.711 mu-law at 8 kHz mono, so the accepted formats are:

1. mulaw @ 8000 Hz — passthrough (recommended; zero conversion cost).
2. linear16 at any sample rate — resampled to 8 kHz and mu-law encoded by the provider.

Anything else (mp3, opus, alaw, multi-channel audio) throws a ConfigurationError telling you how to reconfigure the TTS, e.g. new DeepgramTTS({ options: { encoding: 'mulaw', sampleRate: 8000 } }).

Throws

ConfigurationError If the TTS format cannot be converted to mu-law 8 kHz mono.

Implementation of

AudioOutputProvider.configure

detach()

detach(): void;

Defined in: src/providers/io/twilio/TwilioMediaStream.ts:506

Detach the current socket without closing it.

Returns

void

Remarks

Removes the provider’s message/close listeners from the socket and forgets it. Pending flushes are settled as cancelled. The socket remains open — closing it is the application’s responsibility. Safe to call when no socket is attached.

See

attach()

dispose()

dispose(): Promise<void>;

Defined in: src/providers/io/twilio/TwilioMediaStream.ts:384

Dispose of the provider and release all state.

Returns

Promise<void>

Remarks

Detaches the current socket (removing the provider’s listeners), settles any pending flushes, and resets all call/input/output state. The socket itself is not closed — the application owns its lifecycle. After disposal the provider can be re-initialized.

Implementation of

AudioOutputProvider.dispose

enqueue()

enqueue(chunk): void;

Defined in: src/providers/io/twilio/TwilioMediaStream.ts:859

Enqueue a TTS audio chunk for delivery into the call.

Parameters

Returns

void

Remarks

The chunk is converted per the format set by configure() (mu-law passthrough, or linear16 → resample to 8 kHz → mu-law) and sent immediately as a Twilio media message:

{"event":"media","streamSid":"MZ...","media":{"payload":"<base64 mu-law>"}}

Twilio buffers outbound audio server-side and plays it in real time, so no local pacing is needed. Chunks enqueued before the stream’s start message arrives (no streamSid yet), before a socket is attached, or after the call ends are dropped with a warning. Fires onPlaybackStart on the first chunk after idle.

If chunk.metadata is present it overrides the configured format for that chunk. When configure() was never called, mu-law @ 8 kHz passthrough is assumed.

Implementation of

AudioOutputProvider.enqueue

flush()

flush(): Promise<void>;

Defined in: src/providers/io/twilio/TwilioMediaStream.ts:901

Wait until Twilio has played all audio enqueued so far.

Returns

Promise<void>

Remarks

Sends a mark message ({"event":"mark","streamSid":"MZ...","mark":{"name":"cv-<n>"}}) after the batch of media. Twilio echoes the mark back once call playback reaches it, which resolves the returned promise and — when no other marks are outstanding — fires onPlaybackEnd.

Resolves immediately when nothing is playing or no stream is active. A pending flush also resolves (as cancelled) on barge-in (stop()), remote hangup, detach, or dispose, so the pipeline never hangs on a dead call.

Implementation of

AudioOutputProvider.flush

getCallSid()

getCallSid(): string | null;

Defined in: src/providers/io/twilio/TwilioMediaStream.ts:560

Get the Twilio call SID for the current call.

Returns

string | null

The CA... call SID, or null before the stream starts.

Remarks

Captured from Twilio’s start message. Useful for correlating the media stream with Twilio REST API operations on the call (e.g. redirects, recordings, hangup).

getCustomParameters()

getCustomParameters(): Record<string, string>;

Defined in: src/providers/io/twilio/TwilioMediaStream.ts:580

Get the custom parameters declared on the TwiML <Stream> element.

Returns

Record<string, string>

A copy of the custom parameters map (empty before start).

Remarks

TwiML <Parameter name="..." value="..."/> children of <Stream> are delivered in Twilio’s start message and exposed here — handy for passing per-call context (user id, session token) into the voice agent.

Example

// TwiML: <Stream url="..."><Parameter name="userId" value="42"/></Stream>
const { userId } = twilio.getCustomParameters();

getMetadata()

getMetadata(): AudioMetadata;

Defined in: src/providers/io/twilio/TwilioMediaStream.ts:775

Get the audio format metadata for caller audio.

Returns

AudioMetadata

Metadata describing mu-law 8 kHz mono audio.

Remarks

Twilio Media Streams always deliver G.711 mu-law at 8 kHz mono (audio/x-mulaw). The pipeline uses this metadata to auto-configure the STT provider — e.g. Deepgram receives encoding=mulaw&sample_rate=8000. bitDepth is omitted because mu-law samples are companded, not linear.

Implementation of

AudioInputProvider.getMetadata

getStreamSid()

getStreamSid(): string | null;

Defined in: src/providers/io/twilio/TwilioMediaStream.ts:546

Get the Twilio stream SID for the current call.

Returns

string | null

The MZ... stream SID, or null before the stream starts.

Remarks

Captured from Twilio’s start message. null until the start message arrives (i.e. immediately after attach).

initialize()

initialize(): Promise<void>;

Defined in: src/providers/io/twilio/TwilioMediaStream.ts:369

Initialize the provider, making it ready to accept a socket.

Returns

Promise<void>

Remarks

A no-op beyond setting the initialized flag — the WebSocket is created by Twilio and accepted by your server, not by the provider. If already initialized, this is a no-op.

Implementation of

AudioOutputProvider.initialize

isActive()

isActive(): boolean;

Defined in: src/providers/io/twilio/TwilioMediaStream.ts:745

Check whether the input side is actively emitting audio.

Returns

boolean

true when started and not paused.

Implementation of

AudioInputProvider.isActive

isPlaying()

isPlaying(): boolean;

Defined in: src/providers/io/twilio/TwilioMediaStream.ts:925

Check whether audio is currently being delivered into the call.

Returns

boolean

true between the first enqueued chunk and the point where all flush marks have been echoed (or playback was stopped).

Implementation of

AudioOutputProvider.isPlaying

isReady()

isReady(): boolean;

Defined in: src/providers/io/twilio/TwilioMediaStream.ts:408

Check whether the provider has been initialized.

Returns

boolean

true when initialize has completed and dispose has not yet been called.

Implementation of

AudioOutputProvider.isReady

onAudio()

onAudio(callback): void;

Defined in: src/providers/io/twilio/TwilioMediaStream.ts:760

Register a callback to receive caller audio chunks.

Parameters

Returns

void

Remarks

Chunks contain raw G.711 mu-law bytes at 8 kHz mono, exactly as decoded from Twilio’s base64 media payloads. Only one callback can be registered at a time; subsequent calls replace the previous callback. Must be called before start().

Implementation of

AudioInputProvider.onAudio

onCallEnded()

onCallEnded(callback): void;

Defined in: src/providers/io/twilio/TwilioMediaStream.ts:621

Register a callback invoked when the call ends.

Parameters

Returns

void

Remarks

Fires once per call, when Twilio sends its stop message (caller hung up or the TwiML moved on) or when the socket closes without one. Use it to tear down the pipeline for this call.

Example

twilio.onCallEnded(async () => {
  await agent.dispose();
});

onDtmf()

onDtmf(callback): void;

Defined in: src/providers/io/twilio/TwilioMediaStream.ts:600

Register a callback for DTMF digits pressed by the caller.

Parameters

Returns

void

Remarks

Twilio delivers keypad presses as dtmf messages on the media stream. Only one callback can be registered; subsequent calls replace it.

Example

twilio.onDtmf((digit) => {
  if (digit === '0') transferToHuman();
});

onPlaybackEnd()

onPlaybackEnd(callback): void;

Defined in: src/providers/io/twilio/TwilioMediaStream.ts:948

Register a callback invoked when the playback queue fully drains.

Parameters

Returns

void

Remarks

Fires when the final outstanding flush mark is echoed by Twilio, or when playback is cut short by barge-in (stop()) or remote hangup.

Implementation of

AudioOutputProvider.onPlaybackEnd

onPlaybackError()

onPlaybackError(callback): void;

Defined in: src/providers/io/twilio/TwilioMediaStream.ts:960

Register a callback for playback delivery errors.

Parameters

Returns

void

Remarks

Fires when a chunk cannot be converted or the socket send() throws.

Implementation of

AudioOutputProvider.onPlaybackError

onPlaybackStart()

onPlaybackStart(callback): void;

Defined in: src/providers/io/twilio/TwilioMediaStream.ts:934

Register a callback invoked when audio delivery begins after idle.

Parameters

Returns

void

Implementation of

AudioOutputProvider.onPlaybackStart

pause()

pause(): void;

Defined in: src/providers/io/twilio/TwilioMediaStream.ts:723

Temporarily pause input emission without detaching the socket.

Returns

void

Remarks

Applies to the input side only — the turn-taking controller calls this to mute capture while the agent speaks. Outbound audio delivery is unaffected (Twilio buffers it server-side and cannot pause playback). Caller audio received while paused is silently dropped. Resume with resume().

Implementation of

AudioOutputProvider.pause

resume()

resume(): void;

Defined in: src/providers/io/twilio/TwilioMediaStream.ts:734

Resume input emission after a pause.

Returns

void

See

pause

Implementation of

AudioOutputProvider.resume

start()

start(): void;

Defined in: src/providers/io/twilio/TwilioMediaStream.ts:636

Start emitting caller audio chunks.

Returns

void

Remarks

After calling start(), inbound Twilio media messages are decoded and delivered to the callback registered with onAudio(). Media received while the provider is stopped or paused is silently dropped.

Implementation of

AudioInputProvider.start

stop()

stop(): void;

Defined in: src/providers/io/twilio/TwilioMediaStream.ts:663

Stop the provider — barge-in while the agent is speaking, input halt otherwise.

Returns

void

Remarks

As a duplex provider, the same stop() is invoked by CompositeVoice for two different reasons, so the provider dispatches on playback state:

• Barge-in (output.stop()) — while agent audio is playing or marks are outstanding, a clear message ({"event":"clear","streamSid":"MZ..."}) is sent so Twilio immediately drops any audio it has buffered but not yet played. Outstanding marks are settled as cancelled (their flush() promises resolve so the pipeline never hangs) and the late mark echoes are ignored. Caller audio capture is deliberately not halted: barge-in happens because the caller is speaking, and that speech must keep flowing to STT.
• Stop listening (input.stop()) — when no agent audio is playing and no marks are outstanding, caller audio is no longer emitted (restart with start()).

Fires onPlaybackEnd if playback was in progress.

Implementation of

AudioOutputProvider.stop

stopCapture()

stopCapture(): void;

Defined in: src/providers/io/twilio/TwilioMediaStream.ts:708

Stop emitting caller audio, leaving any playback alone.

Returns

void

Remarks

The pipeline calls this when it means “stop listening”. Restart with start().

Implementation of

AudioInputProvider.stopCapture

stopPlayback()

stopPlayback(): void;

Defined in: src/providers/io/twilio/TwilioMediaStream.ts:688

Stop playback, leaving caller audio flowing.

Returns

void

Remarks

The pipeline calls this for barge-in, so the provider never has to infer which side was meant. Sends Twilio a clear so it drops audio already buffered on its side, settles outstanding marks as cancelled (their flush() promises resolve, so the pipeline cannot hang), and ignores the late mark echoes that follow.

Capture is deliberately untouched: barge-in fires because the caller is speaking, and that speech has to reach STT. Safe when nothing is playing — barge-in is also raised while the agent is still thinking.

Implementation of

AudioOutputProvider.stopPlayback