LLM Class Overview

Mastra provides direct support for Large Language Models (LLMs) through the LLM class. The LLM class allows you to interact with various language models seamlessly, enabling you to generate text, handle conversations, and more. This guide covers:

• How to initialize the LLM class.
• Supported models and providers.
• Using the generate function.
• Message formats in generate.
• Output formats in generate.

Initializing the LLM Class

To start using the LLM class, you need to initialize it with the desired model configuration. Here’s how you can do it:

import { Mastra } from "@mastra/core";
 
const mastra = new Mastra();
 
const llm = mastra.LLM({
  provider: "OPEN_AI",
  name: "gpt-4o-mini",
});

This initialization allows telemetry to pass through to the LLM, providing insights into model usage and performance.

Note: You can find more details about the model configuration options in the ModelConfig class reference.

Supported Models and Providers

Mastra supports major LLM providers out of the box, plus additional providers through AI SDK integrations. Custom providers can also be added via the Portkey service.

Most Popular Providers and Models

A full list of supported models can be found here.

The generate Function

The main function you’ll use with the LLM class is generate. It allows you to send messages to the language model and receive responses. The generate function takes:

• messages: The first parameter, which can be a string, an array of strings, or an array of message objects.
• options: The second parameter, which includes additional configurations like streaming, schemas for structured output, etc.

This design covers all potential use cases and is extensible to multi-modal interactions in the future.

Message Formats in generate

The generate function supports three types of message formats:

1. Simple String

You can pass a single string as the message:

const response = await llm.generate("Tell me a joke.");

2. Array of Strings

You can provide an array of strings, which will be converted into user messages:

const response = await llm.generate([
  "Hello!",
  "Can you explain quantum mechanics?",
]);

3. Detailed Message Objects

For finer control, you can pass an array of message objects, specifying the role and content:

const response = await llm.generate([
  { role: "system", content: "You are a helpful assistant." },
  { role: "user", content: "What is the meaning of life?" },
]);

Output Formats in generate

The generate function supports four types of output formats:

1. Simple Text Generation

Receive a basic text response from the model:

const response = await llm.generate("What is AI?");
 
console.log(response.text);

2. Structured Output

Request a structured response by providing a schema. This is useful when you need the output in a specific format:

import { z } from "zod";
 
const mySchema = z.object({
  definition: z.string(),
  examples: z.array(z.string()),
});
 
const response = await llm.generate(
  "Define machine learning and give examples.",
  {
    output: mySchema,
  },
);
 
console.log(response.object);

3. Streaming Text

Stream the response in real-time, which is useful for handling longer outputs or providing immediate feedback to users:

const stream = await llm.stream("Tell me a story about a brave knight.");
 
for await (const chunk of stream.textStream) {
  process.stdout.write(chunk);
}

4. Streaming Structured Output

Stream a structured response using a schema:

const stream = await llm.stream("Provide live weather data.", {
  output: mySchema,
});
 
for await (const chunk of stream.textStream) {
  console.log(chunk);
}

Additional Notes

• Telemetry: Initializing the LLM class through Mastra allows telemetry data to pass through, enabling better monitoring and debugging.
• Extensibility: The design of the generate function and message formats makes it future-proof and extensible for multi-modal interactions.