LLMGenerationOptions

Options for controlling a single LLM generation request.

Defined in: src/core/types/providers.ts:803

Options for controlling a single LLM generation request.

Remarks

These options override the provider-level LLMProviderConfig settings for a specific generation call. They are passed to generate and generateFromMessages.

The signal property is particularly important for the eager LLM pipeline, where speculative generations may need to be cancelled.

Example

const controller = new AbortController();

const options: LLMGenerationOptions = {
  temperature: 0.5,
  maxTokens: 100,
  signal: controller.signal,
};

const stream = await llmProvider.generate('Hello!', options);
// Cancel if needed:
controller.abort();

See

LLMProviderConfig for provider-level defaults
LLMProvider for the provider interface that accepts these options

Properties

Property	Type	Description	Defined in
`extra?`	`Record`<`string`, `unknown`>	Additional provider-specific options. Remarks Allows passing through options that are specific to a particular LLM provider without extending this interface.	src/core/types/providers.ts:845
`maxTokens?`	`number`	Override the provider’s default max tokens for this generation. See LLMProviderConfig.maxTokens	src/core/types/providers.ts:816
`signal?`	`AbortSignal`	AbortSignal for cancelling an in-flight generation. Remarks Providers that support cancellation (Anthropic, OpenAI) will stop yielding tokens and throw an `AbortError` when this signal fires. Used by CompositeVoice for the eager/preflight pipeline to cancel speculative generations when the confirmed text differs. See EagerLLMConfig for the eager pipeline configuration	src/core/types/providers.ts:836
`stopSequences?`	`string`[]	Override the provider’s default stop sequences for this generation. See LLMProviderConfig.stopSequences	src/core/types/providers.ts:823
`temperature?`	`number`	Override the provider’s default temperature for this generation. See LLMProviderConfig.temperature	src/core/types/providers.ts:809