## Model routing
When creating agents in Mastra, you can specify any AI SDK-supported model:
```typescript
import { openai } from "@ai-sdk/openai";
import { Agent } from "@mastra/core/agent";
const agent = new Agent({
name: "WeatherAgent",
instructions: "Instructions for the agent...",
model: openai("gpt-4-turbo"), // Model comes directly from AI SDK
});
const result = await agent.generate("What is the weather like?");
```
## AI SDK Hooks
Mastra is compatible with AI SDK's hooks for seamless frontend integration:
### useChat
The `useChat` hook enables real-time chat interactions in your frontend application
- Works with agent data streams i.e. `.toDataStreamResponse()`
- The useChat `api` defaults to `/api/chat`
- Works with the Mastra REST API agent stream endpoint `{MASTRA_BASE_URL}/agents/:agentId/stream` for data streams,
i.e. no structured output is defined.
```typescript filename="app/api/chat/route.ts" copy
import { mastra } from '@/src/mastra';
export async function POST(req: Request) {
const { messages } = await req.json();
const myAgent = mastra.getAgent('weatherAgent');
const stream = await myAgent.stream(messages);
return stream.toDataStreamResponse();
}
```
```typescript copy
import { useChat } from '@ai-sdk/react';
export function ChatComponent() {
const { messages, input, handleInputChange, handleSubmit } = useChat({
api: '/path-to-your-agent-stream-api-endpoint'
});
return (
{messages.map(m => (
);
}
```
> **Gotcha**: When using `useChat` with agent memory functionality, make sure to check out the [Agent Memory section](/docs/agents/01-agent-memory#usechat) for important implementation details.
### useCompletion
For single-turn completions, use the `useCompletion` hook:
- Works with agent data streams i.e. `.toDataStreamResponse()`
- The useCompletion `api` defaults to `/api/completion`
- Works with the Mastra REST API agent stream endpoint `{MASTRA_BASE_URL}/agents/:agentId/stream` for data streams,
i.e. no structured output is defined.
```typescript filename="app/api/completion/route.ts" copy
import { mastra } from '@/src/mastra';
export async function POST(req: Request) {
const { messages } = await req.json();
const myAgent = mastra.getAgent('weatherAgent');
const stream = await myAgent.stream(messages);
return stream.toDataStreamResponse();
}
```
```typescript
import { useCompletion } from "@ai-sdk/react";
export function CompletionComponent() {
const {
completion,
input,
handleInputChange,
handleSubmit,
} = useCompletion({
api: '/path-to-your-agent-stream-api-endpoint'
});
return (
{m.role}: {m.content}
))}
Completion result: {completion}
{object?.content &&
);
}
```
## Tool Calling
### AI SDK Tool Format
Mastra supports tools created using the AI SDK format, so you can use
them directly with Mastra agents. See our tools doc on [Vercel AI SDK Tool Format
](/docs/agents/02-adding-tools#vercel-ai-sdk-tool-format) for more details.
### Client-side tool calling
Mastra leverages AI SDK's tool calling, so what applies in AI SDK applies here still.
[Agent Tools](/docs/agents/02-adding-tools) in Mastra are 100% percent compatible with AI SDK tools.
Mastra tools also expose an optional `execute` async function. It is optional because you might want to forward tool calls to the client or to a queue instead of executing them in the same process.
One way to then leverage client-side tool calling is to use the `@ai-sdk/react` `useChat` hook's `onToolCall` property for
client-side tool execution
---
title: "Installing Mastra Locally | Getting Started | Mastra Docs"
description: Guide on installing Mastra and setting up the necessary prerequisites for running it with various LLM providers.
---
import { Callout, Steps, Tabs } from "nextra/components";
import YouTube from "../../../components/youtube";
# Installing Mastra Locally
Source: https://mastra.ai/docs/getting-started/installation
To run Mastra, you need access to an LLM. Typically, you'll want to get an API key from an LLM provider such as [OpenAI](https://platform.openai.com/), [Anthropic](https://console.anthropic.com/settings/keys), or [Google Gemini](https://ai.google.dev/gemini-api/docs). You can also run Mastra with a local LLM using [Ollama](https://ollama.ai/).
## Prerequisites
- Node.js `v20.0` or higher
- Access to a [supported large language model (LLM)](/docs/reference/llm/providers-and-models)
## Automatic Installation
{object.content}
}
### Other Supported Providers
For a complete list of supported observability providers and their configuration details, see the [Observability Providers reference](../reference/observability/providers/).
### Framework-Specific Setup
For Next.js applications, see our [Next.js Tracing](/docs/observability/nextjs-tracing) guide.
---
title: Chunking and Embedding Documents | RAG | Mastra Docs
description: Guide on chunking and embedding documents in Mastra for efficient processing and retrieval.
---
## Chunking and Embedding Documents
Source: https://mastra.ai/docs/rag/chunking-and-embedding
Before processing, create a MDocument instance from your content. You can initialize it from various formats:
```ts showLineNumbers copy
const docFromText = MDocument.fromText("Your plain text content...");
const docFromHTML = MDocument.fromHTML("Your HTML content...");
const docFromMarkdown = MDocument.fromMarkdown("# Your Markdown content...");
const docFromJSON = MDocument.fromJSON(`{ "key": "value" }`);
```
## Step 1: Document Processing
Use `chunk` to split documents into manageable pieces. Mastra supports multiple chunking strategies optimized for different document types:
- `recursive`: Smart splitting based on content structure
- `character`: Simple character-based splits
- `token`: Token-aware splitting
- `markdown`: Markdown-aware splitting
- `html`: HTML structure-aware splitting
- `json`: JSON structure-aware splitting
- `latex`: LaTeX structure-aware splitting
Here's an example of how to use the `recursive` strategy:
```ts showLineNumbers copy
const chunks = await doc.chunk({
strategy: "recursive",
size: 512,
overlap: 50,
separator: "\n",
extract: {
metadata: true, // Optionally extract metadata
},
});
```
**Note:** Metadata extraction may use LLM calls, so ensure your API key is set.
We go deeper into chunking strategies in our [chunk documentation](/docs/reference/rag/chunk.mdx).
## Step 2: Embedding Generation
Transform chunks into embeddings using your preferred provider. Mastra supports both OpenAI and Cohere embeddings:
### Using OpenAI
```ts showLineNumbers copy
import { openai } from "@ai-sdk/openai";
import { embedMany } from "ai";
const { embeddings } = await embedMany({
model: openai.embedding('text-embedding-3-small'),
values: chunks.map(chunk => chunk.text),
});
```
### Using Cohere
```ts showLineNumbers copy
import { embedMany } from 'ai';
import { cohere } from '@ai-sdk/cohere';
const { embeddings } = await embedMany({
model: cohere.embedding('embed-english-v3.0'),
values: chunks.map(chunk => chunk.text),
});
```
The embedding functions return vectors, arrays of numbers representing the semantic meaning of your text, ready for similarity searches in your vector database.
## Example: Complete Pipeline
Here's an example showing document processing and embedding generation with both providers:
```ts showLineNumbers copy
import { embedMany } from "ai";
import { openai } from "@ai-sdk/openai";
import { cohere } from "@ai-sdk/cohere";
import { MDocument } from "@mastra/rag";
// Initialize document
const doc = MDocument.fromText(`
Climate change poses significant challenges to global agriculture.
Rising temperatures and changing precipitation patterns affect crop yields.
`);
// Create chunks
const chunks = await doc.chunk({
strategy: "recursive",
size: 256,
overlap: 50,
});
// Generate embeddings with OpenAI
const { embeddings: openAIEmbeddings } = await embedMany({
model: openai.embedding('text-embedding-3-small'),
values: chunks.map(chunk => chunk.text),
});
// OR
// Generate embeddings with Cohere
const { embeddings: cohereEmbeddings } = await embedMany({
model: cohere.embedding('embed-english-v3.0'),
values: chunks.map(chunk => chunk.text),
});
// Store embeddings in your vector database
await vectorStore.upsert({
indexName: "embeddings",
vectors: embeddings,
});
```
This example demonstrates how to process a document, split it into chunks, generate embeddings with both OpenAI and Cohere, and store the results in a vector database.
For more examples of different chunking strategies and embedding configurations, see:
- [Adjust Chunk Size](/docs/reference/rag/chunk.mdx#adjust-chunk-size)
- [Adjust Chunk Delimiters](/docs/reference/rag/chunk.mdx#adjust-chunk-delimiters)
- [Embed Text with Cohere](/docs/reference/rag/embeddings.mdx#using-cohere)
---
title: RAG (Retrieval-Augmented Generation) in Mastra | Mastra Docs
description: Overview of Retrieval-Augmented Generation (RAG) in Mastra, detailing its capabilities for enhancing LLM outputs with relevant context.
---
# RAG (Retrieval-Augmented Generation) in Mastra
Source: https://mastra.ai/docs/rag/overview
RAG in Mastra helps you enhance LLM outputs by incorporating relevant context from your own data sources, improving accuracy and grounding responses in real information.
Mastra's RAG system provides:
- Standardized APIs to process and embed documents
- Support for multiple vector stores
- Chunking and embedding strategies for optimal retrieval
- Observability for tracking embedding and retrieval performance
## Example
To implement RAG, you process your documents into chunks, create embeddings, store them in a vector database, and then retrieve relevant context at query time.
```ts showLineNumbers copy
import { embedMany } from "ai";
import { openai } from "@ai-sdk/openai";
import { PgVector } from "@mastra/pg";
import { MDocument } from "@mastra/rag";
import { z } from "zod";
// 1. Initialize document
const doc = MDocument.fromText(`Your document text here...`);
// 2. Create chunks
const chunks = await doc.chunk({
strategy: "recursive",
size: 512,
overlap: 50,
});
// 3. Generate embeddings; we need to pass the text of each chunk
const { embeddings } = await embedMany({
values: chunks.map(chunk => chunk.text),
model: openai.embedding("text-embedding-3-small"),
});
// 4. Store in vector database
const pgVector = new PgVector(process.env.POSTGRES_CONNECTION_STRING);
await pgVector.upsert({
indexName: "embeddings",
vectors: embeddings,
}); // using an index name of 'embeddings'
// 5. Query similar chunks
const results = await pgVector.query({
indexName: "embeddings",
queryVector: queryVector,
topK: 3,
}); // queryVector is the embedding of the query
console.log("Similar chunks:", results);
```
This example shows the essentials: initialize a document, create chunks, generate embeddings, store them, and query for similar content.
## Document Processing
The basic building block of RAG is document processing. Documents can be chunked using various strategies (recursive, sliding window, etc.) and enriched with metadata. See the [chunking and embedding doc](./chunking-and-embedding.mdx).
## Vector Storage
Mastra supports multiple vector stores for embedding persistence and similarity search, including pgvector, Pinecone, and Qdrant. See the [vector database doc](./vector-databases.mdx).
## Observability and Debugging
Mastra's RAG system includes observability features to help you optimize your retrieval pipeline:
- Track embedding generation performance and costs
- Monitor chunk quality and retrieval relevance
- Analyze query patterns and cache hit rates
- Export metrics to your observability platform
See the [OTel Configuration](../reference/observability/otel-config.mdx) page for more details.
## More resources
- [Chain of Thought RAG Example](../../examples/rag/usage/cot-rag.mdx)
- [All RAG Examples](../../examples/) (including different chunking strategies, embedding models, and vector stores)
---
title: "Retrieval, Semantic Search, Reranking | RAG | Mastra Docs"
description: Guide on retrieval processes in Mastra's RAG systems, including semantic search, filtering, and re-ranking.
---
import { Tabs } from "nextra/components";
## Retrieval in RAG Systems
Source: https://mastra.ai/docs/rag/retrieval
After storing embeddings, you need to retrieve relevant chunks to answer user queries.
Mastra provides flexible retrieval options with support for semantic search, filtering, and re-ranking.
## How Retrieval Works
1. The user's query is converted to an embedding using the same model used for document embeddings
2. This embedding is compared to stored embeddings using vector similarity
3. The most similar chunks are retrieved and can be optionally:
- Filtered by metadata
- Re-ranked for better relevance
- Processed through a knowledge graph
## Basic Retrieval
The simplest approach is direct semantic search. This method uses vector similarity to find chunks that are semantically similar to the query:
```ts showLineNumbers copy
import { openai } from "@ai-sdk/openai";
import { embed } from "ai";
import { PgVector } from "@mastra/pg";
// Convert query to embedding
const { embedding } = await embed({
value: "What are the main points in the article?",
model: openai.embedding('text-embedding-3-small'),
});
// Query vector store
const pgVector = new PgVector(process.env.POSTGRES_CONNECTION_STRING);
const results = await pgVector.query({
indexName: "embeddings",
queryVector: embedding,
topK: 10,
});
// Display results
console.log(results);
```
Results include both the text content and a similarity score:
```ts showLineNumbers copy
[
{
text: "Climate change poses significant challenges...",
score: 0.89,
metadata: { source: "article1.txt" }
},
{
text: "Rising temperatures affect crop yields...",
score: 0.82,
metadata: { source: "article1.txt" }
}
// ... more results
]
```
For an example of how to use the basic retrieval method, see the [Retrieve Results](../../examples/rag/query/retrieve-results.mdx) example.
## Advanced Retrieval options
### Metadata Filtering
Filter results based on metadata fields to narrow down the search space. This is useful when you have documents from different sources, time periods, or with specific attributes. Mastra provides a unified MongoDB-style query syntax that works across all supported vector stores.
For detailed information about available operators and syntax, see the [Metadata Filters Reference](/docs/reference/rag/metadata-filters).
Basic filtering examples:
```ts showLineNumbers copy
// Simple equality filter
const results = await pgVector.query({
indexName: "embeddings",
queryVector: embedding,
topK: 10,
filter: {
source: "article1.txt"
}
});
// Numeric comparison
const results = await pgVector.query({
indexName: "embeddings",
queryVector: embedding,
topK: 10,
filter: {
price: { $gt: 100 }
}
});
// Multiple conditions
const results = await pgVector.query({
indexName: "embeddings",
queryVector: embedding,
topK: 10,
filter: {
category: "electronics",
price: { $lt: 1000 },
inStock: true
}
});
// Array operations
const results = await pgVector.query({
indexName: "embeddings",
queryVector: embedding,
topK: 10,
filter: {
tags: { $in: ["sale", "new"] }
}
});
// Logical operators
const results = await pgVector.query({
indexName: "embeddings",
queryVector: embedding,
topK: 10,
filter: {
$or: [
{ category: "electronics" },
{ category: "accessories" }
],
$and: [
{ price: { $gt: 50 } },
{ price: { $lt: 200 } }
]
}
});
```
Common use cases for metadata filtering:
- Filter by document source or type
- Filter by date ranges
- Filter by specific categories or tags
- Filter by numerical ranges (e.g., price, rating)
- Combine multiple conditions for precise querying
- Filter by document attributes (e.g., language, author)
For an example of how to use metadata filtering, see the [Hybrid Vector Search](../../examples/rag/query/hybrid-vector-search.mdx) example.
### Vector Query Tool
Sometimes you want to give your agent the ability to query a vector database directly. The Vector Query Tool allows your agent to be in charge of retrieval decisions, combining semantic search with optional filtering and reranking based on the agent's understanding of the user's needs.
```ts showLineNumbers copy
const vectorQueryTool = createVectorQueryTool({
vectorStoreName: 'pgVector',
indexName: 'embeddings',
model: openai.embedding('text-embedding-3-small'),
});
```
When creating the tool, pay special attention to the tool's name and description - these help the agent understand when and how to use the retrieval capabilities. For example, you might name it "SearchKnowledgeBase" and describe it as "Search through our documentation to find relevant information about X topic."
This is particularly useful when:
- Your agent needs to dynamically decide what information to retrieve
- The retrieval process requires complex decision-making
- You want the agent to combine multiple retrieval strategies based on context
For detailed configuration options and advanced usage, see the [Vector Query Tool Reference](/docs/reference/tools/vector-query-tool).
### Vector Store Prompts
Vector store prompts define query patterns and filtering capabilities for each vector database implementation.
When implementing filtering, these prompts are required in the agent's instructions to specify valid operators and syntax for each vector store implementation.