Calling LLMs

Mastra has direct LLM support through the LLM class. Note that this class is pretty barebones: you can specify a model and a user prompt, but you do not have the ability to use system prompts, tools, or workflows.

Basic Setup

import { Mastra } from '@mastra/core';
 
async function main() {
  // Configure your model
 
  const mastra = new Mastra();
 
  const modelConfig: ModelConfig = {
    provider: 'OPEN_AI',
    name: 'gpt-4o',
    toolChoice: 'auto'
  };
 
  const llm = mastra.llm
 
  const response = await llm.text({
    messages: [{
      role: 'user',
      content: 'What is machine learning?'
    }],
    model: modelConfig
  });
  console.log(response.text);
}
 
main();

Available Models

You can choose from several providers and models:

• OpenAI models: ‘gpt-4’, ‘gpt-4-turbo’, ‘gpt-3.5-turbo’, ‘gpt-4o’, ‘gpt-4o-mini’

• Anthropic models: ‘claude-3-5-sonnet-20240620’, ‘claude-3-opus-20240229’, ‘claude-3-sonnet-20240229’, ‘claude-3-haiku-20240307’

• Groq models: ‘llama3-groq-70b-8192-tool-use-preview’, ‘llama3-groq-8b-8192-tool-use-preview’

Usage Examples

Text Generation

const response = await llm.text({
  messages: [{
    role: 'user',
    content: 'What is machine learning?'
  }],
  model: modelConfig
});
 
console.log(response.text);

Streaming Responses

const stream = await llm.stream({
  messages: [{
    role: 'system',
    content: 'You are a helpful assistant',
  }, {
    role: 'user',
    content: 'Explain quantum computing'
  }],
  model: modelConfig,
  onStepFinish: (step) => {
    console.log('Step completed:', step);
  },
  maxSteps: 3
});

Environment Setup

Make sure to set your API keys. If you don’t have an API key for an LLM provider, you can get one from the following services:

• OpenAI. Env variable: OPENAI_API_KEY
• Anthropic. Env variable: ANTHROPIC_API_KEY
• Google Gemini. Env variable: GOOGLE_GENERATIVE_AI_API_KEY

If you don’t have an account with these providers, you can sign up and get an API key. OpenAI and Anthropic require a credit card to get an API key. Gemini does not and has a generous free tier for its API.