`stream()`

The stream() method enables real-time streaming of responses from the language model. This method accepts messages and an optional options object as parameters, similar to generate().

Parameters

`messages`

The messages parameter can be:

A single string
An array of strings
An array of message objects with role and content properties

Message Object Structure

interface Message {
  role: 'system' | 'user' | 'assistant';
  content: string;
}

`options` (Optional)

An optional object that can include:

output?:

string | JSONSchema7 | ZodSchema

Defines the output format. Can be "text" or a schema for structured output.

onFinish?:

(result: string) => Promise<void> | void

Callback function called when streaming is complete.

onStepFinish?:

(step: string) => void

Callback function called after each step during streaming.

maxSteps?:

number

Maximum number of steps allowed during streaming.

tools?:

ToolsInput

Tools available for the LLM to use during streaming.

runId?:

string

Unique identifier for the streaming run, useful for tracing and logging.

Returns

The method returns a promise that resolves to an object containing one or more of the following properties:

textStream?:

AsyncIterable<string>

An async iterable stream of text chunks. Present when output is "text".

objectStream?:

AsyncIterable<object>

An async iterable stream of structured data. Present when a schema is provided.

object?:

Promise<object>

A promise that resolves to the final structured output when using a schema.

Examples

Basic Text Streaming

const stream = await llm.stream("Tell me a story about a brave knight.");
 
for await (const chunk of stream.textStream) {
  process.stdout.write(chunk);
}

Structured Output Streaming

const schema = {
  type: 'object',
  properties: {
    answer: { type: 'number' },
    explanation: { type: 'string' }
  },
  required: ['answer', 'explanation']
};
 
const response = await llm.stream("What is 2+2?", {
  output: schema,
  onFinish: text => console.log("Finished:", text)
});
 
for await (const chunk of response.textStream) {
  console.log(chunk);
}
 
const result = await response.object;
console.log("Final structured result:", result);

generate Chunk

stream()