Upsert Embeddings

After generating embeddings, you need to store them in a database that supports vector similarity search. This example shows how to store embeddings in various vector databases for later retrieval.

PgVector

The PgVector class provides methods to create indexes and insert embeddings into PostgreSQL with the pgvector extension.


import { openai } from "@ai-sdk/openai";
import { PgVector } from "@mastra/pg";
import { MDocument } from "@mastra/rag";
import { embedMany } from "ai";
 
const doc = MDocument.fromText("Your text content...");
 
const chunks = await doc.chunk();
 
const { embeddings } = await embedMany({
  values: chunks.map(chunk => chunk.text),
  model: openai.embedding("text-embedding-3-small"),
});
 
const pgVector = new PgVector(process.env.POSTGRES_CONNECTION_STRING!);
 
await pgVector.createIndex({
  indexName: "test_index",
  dimension: 1536,
});
 
await pgVector.upsert({
  indexName: "test_index",
  vectors: embeddings,
  metadata: chunks?.map((chunk: any) => ({ text: chunk.text })),
});

View Example on GitHub

Pinecone

The PineconeVector class provides methods to create indexes and insert embeddings into Pinecone, a managed vector database service.


import { openai } from '@ai-sdk/openai';
import { PineconeVector } from '@mastra/pinecone';
import { MDocument } from '@mastra/rag';
import { embedMany } from 'ai';
 
const doc = MDocument.fromText('Your text content...');
 
const chunks = await doc.chunk();
 
const { embeddings } = await embedMany({
  values: chunks.map(chunk => chunk.text),
  model: openai.embedding('text-embedding-3-small'),
});
 
const pinecone = new PineconeVector(process.env.PINECONE_API_KEY!);
 
await pinecone.createIndex({
  indexName: 'testindex',
  dimension: 1536,
});
 
await pinecone.upsert({
  indexName: 'testindex',
  vectors: embeddings,
  metadata: chunks?.map(chunk => ({ text: chunk.text })),
});

View Example on GitHub

Qdrant

The QdrantVector class provides methods to create collections and insert embeddings into Qdrant, a high-performance vector database.


import { openai } from '@ai-sdk/openai';
import { QdrantVector } from '@mastra/qdrant';
import { MDocument } from '@mastra/rag';
import { embedMany } from 'ai';
 
const doc = MDocument.fromText('Your text content...');
 
const chunks = await doc.chunk();
 
const { embeddings } = await embedMany({
  values: chunks.map(chunk => chunk.text),
  model: openai.embedding('text-embedding-3-small'),
  maxRetries: 3,
});
 
const qdrant = new QdrantVector(
  process.env.QDRANT_URL,
  process.env.QDRANT_API_KEY,
);
 
await qdrant.createIndex({
  indexName: 'test_collection',
  dimension: 1536,
});
 
await qdrant.upsert({
  indexName: 'test_collection',
  vectors: embeddings,
  metadata: chunks?.map(chunk => ({ text: chunk.text })),
});

Chroma

The ChromaVector class provides methods to create collections and insert embeddings into Chroma, an open-source embedding database.


import { openai } from '@ai-sdk/openai';
import { ChromaVector } from '@mastra/chroma';
import { MDocument } from '@mastra/rag';
import { embedMany } from 'ai';
 
const doc = MDocument.fromText('Your text content...');
 
const chunks = await doc.chunk();
 
const { embeddings } = await embedMany({
  values: chunks.map(chunk => chunk.text),
  model: openai.embedding('text-embedding-3-small'),
});
 
const chroma = new ChromaVector({
  path: "path/to/chroma/db",
});
 
await chroma.createIndex({
  indexName: 'test_collection',
  dimension: 1536,
});
 
await chroma.upsert({
  indexName: 'test_collection',
  vectors: embeddings,
  metadata: chunks.map(chunk => ({ text: chunk.text })),
  documents: chunks.map(chunk => chunk.text),
});

View Example on GitHub

Astra DB

he AstraVector class provides methods to create collections and insert embeddings into DataStax Astra DB, a cloud-native vector database.


import { openai } from '@ai-sdk/openai';
import { AstraVector } from '@mastra/astra';
import { MDocument } from '@mastra/rag';
import { embedMany } from 'ai';
 
const doc = MDocument.fromText('Your text content...');
 
const chunks = await doc.chunk();
 
const { embeddings } = await embedMany({
  model: openai.embedding('text-embedding-3-small'),
  values: chunks.map(chunk => chunk.text),
});
 
const astra = new AstraVector({
  token: process.env.ASTRA_DB_TOKEN,
  endpoint: process.env.ASTRA_DB_ENDPOINT,
  keyspace: process.env.ASTRA_DB_KEYSPACE,
});
 
await astra.createIndex({
  indexName: 'test_collection',
  dimension: 1536,
});
 
await astra.upsert({
  indexName: 'test_collection',
  vectors: embeddings,
  metadata: chunks?.map(chunk => ({ text: chunk.text })),
});

LibSQL

The LibSQLVector class provides methods to create collections and insert embeddings into LibSQL, a fork of SQLite with vector extensions.


import { openai } from "@ai-sdk/openai";
import { LibSQLVector } from "@mastra/core/vector/libsql";
import { MDocument } from "@mastra/rag";
import { embedMany } from "ai";
 
const doc = MDocument.fromText("Your text content...");
 
const chunks = await doc.chunk();
 
const { embeddings } = await embedMany({
  values: chunks.map((chunk) => chunk.text),
  model: openai.embedding("text-embedding-3-small"),
});
 
const libsql = new LibSQLVector({
  connectionUrl: process.env.DATABASE_URL,
  authToken: process.env.DATABASE_AUTH_TOKEN, // Optional: for Turso cloud databases
});
 
await libsql.createIndex({
  indexName: "test_collection",
  dimension: 1536,
});
 
await libsql.upsert({
  indexName: "test_collection",
  vectors: embeddings,
  metadata: chunks?.map((chunk) => ({ text: chunk.text })),
});

View Example on GitHub

Upstash

The UpstashVector class provides methods to create collections and insert embeddings into Upstash Vector, a serverless vector database.


import { openai } from '@ai-sdk/openai';
import { UpstashVector } from '@mastra/upstash';
import { MDocument } from '@mastra/rag';
import { embedMany } from 'ai';
 
const doc = MDocument.fromText('Your text content...');
 
const chunks = await doc.chunk();
 
const { embeddings } = await embedMany({
  values: chunks.map(chunk => chunk.text),
  model: openai.embedding('text-embedding-3-small'),
});
 
const upstash = new UpstashVector({
  url: process.env.UPSTASH_URL,
  token: process.env.UPSTASH_TOKEN,
});
 
await upstash.createIndex({
  indexName: 'test_collection',
  dimension: 1536,
});
 
await upstash.upsert({
  indexName: 'test_collection',
  vectors: embeddings,
  metadata: chunks?.map(chunk => ({ text: chunk.text })),
});

Cloudflare

The CloudflareVector class provides methods to create collections and insert embeddings into Cloudflare Vectorize, a serverless vector database service.


import { openai } from '@ai-sdk/openai';
import { CloudflareVector } from '@mastra/vectorize';
import { MDocument } from '@mastra/rag';
import { embedMany } from 'ai';
 
const doc = MDocument.fromText('Your text content...');
 
const chunks = await doc.chunk();
 
const { embeddings } = await embedMany({
  values: chunks.map(chunk => chunk.text),
  model: openai.embedding('text-embedding-3-small'),
});
 
const vectorize = new CloudflareVector({
  accountId: process.env.CF_ACCOUNT_ID,
  apiToken: process.env.CF_API_TOKEN,
});
 
await vectorize.createIndex({
  indexName: 'test_collection',
  dimension: 1536,
});
 
await vectorize.upsert({
  indexName: 'test_collection',
  vectors: embeddings,
  metadata: chunks?.map(chunk => ({ text: chunk.text })),
});

MongoDB

The MongoDBVector class provides methods to create indexes and insert embeddings into MongoDB with Atlas Search.


import { openai } from "@ai-sdk/openai";
import { MongoDBVector } from "@mastra/mongodb";
import { MDocument } from "@mastra/rag";
import { embedMany } from "ai";
 
const doc = MDocument.fromText("Your text content...");
 
const chunks = await doc.chunk();
 
const { embeddings } = await embedMany({
  values: chunks.map(chunk => chunk.text),
  model: openai.embedding("text-embedding-3-small"),
});
 
const vectorDB = new MongoDBVector({
  uri: process.env.MONGODB_URI!,
  dbName: process.env.MONGODB_DB_NAME!,
});
 
await vectorDB.createIndex({
  indexName: "test_index",
  dimension: 1536,
});
 
await vectorDB.upsert({
  indexName: "test_index",
  vectors: embeddings,
  metadata: chunks?.map((chunk: any) => ({ text: chunk.text })),
});