Upsert Embeddings

After generating embeddings, you need to store them in a database that supports vector similarity search. This example shows how to store embeddings in various vector databases for later retrieval.

The PgVector class provides methods to create indexes and insert embeddings into PostgreSQL with the pgvector extension.

import { openai } from "@ai-sdk/openai";
import { PgVector } from "@mastra/pg";
import { MDocument } from "@mastra/rag";
import { embedMany } from "ai";
 
const doc = MDocument.fromText("Your text content...");
 
const chunks = await doc.chunk();
 
const { embeddings } = await embedMany({
  values: chunks.map(chunk => chunk.text),
  model: openai.embedding("text-embedding-3-small"),
});
 
const pgVector = new PgVector(process.env.POSTGRES_CONNECTION_STRING!);
 
await pgVector.createIndex({
  indexName: "test_index",
  dimension: 1536,
});
 
await pgVector.upsert({
  indexName: "test_index",
  vectors: embeddings,
  metadata: chunks?.map((chunk: any) => ({ text: chunk.text })),
});

View Example on GitHub

The PineconeVector class provides methods to create indexes and insert embeddings into Pinecone, a managed vector database service.

import { openai } from '@ai-sdk/openai';
import { PineconeVector } from '@mastra/pinecone';
import { MDocument } from '@mastra/rag';
import { embedMany } from 'ai';
 
const doc = MDocument.fromText('Your text content...');
 
const chunks = await doc.chunk();
 
const { embeddings } = await embedMany({
  values: chunks.map(chunk => chunk.text),
  model: openai.embedding('text-embedding-3-small'),
});
 
const pinecone = new PineconeVector(process.env.PINECONE_API_KEY!);
 
await pinecone.createIndex({
  indexName: 'testindex',
  dimension: 1536,
});
 
await pinecone.upsert({
  indexName: 'testindex',
  vectors: embeddings,
  metadata: chunks?.map(chunk => ({ text: chunk.text })),
});

View Example on GitHub

The QdrantVector class provides methods to create collections and insert embeddings into Qdrant, a high-performance vector database.

import { openai } from '@ai-sdk/openai';
import { QdrantVector } from '@mastra/qdrant';
import { MDocument } from '@mastra/rag';
import { embedMany } from 'ai';
 
const doc = MDocument.fromText('Your text content...');
 
const chunks = await doc.chunk();
 
const { embeddings } = await embedMany({
  values: chunks.map(chunk => chunk.text),
  model: openai.embedding('text-embedding-3-small'),
  maxRetries: 3,
});
 
const qdrant = new QdrantVector(
  process.env.QDRANT_URL,
  process.env.QDRANT_API_KEY,
);
 
await qdrant.createIndex({
  indexName: 'test_collection',
  dimension: 1536,
});
 
await qdrant.upsert({
  indexName: 'test_collection',
  vectors: embeddings,
  metadata: chunks?.map(chunk => ({ text: chunk.text })),
});

The ChromaVector class provides methods to create collections and insert embeddings into Chroma, an open-source embedding database.

import { openai } from '@ai-sdk/openai';
import { ChromaVector } from '@mastra/chroma';
import { MDocument } from '@mastra/rag';
import { embedMany } from 'ai';
 
const doc = MDocument.fromText('Your text content...');
 
const chunks = await doc.chunk();
 
const { embeddings } = await embedMany({
  values: chunks.map(chunk => chunk.text),
  model: openai.embedding('text-embedding-3-small'),
});
 
const chroma = new ChromaVector({
  path: "path/to/chroma/db",
});
 
await chroma.createIndex({
  indexName: 'test_collection',
  dimension: 1536,
});
 
await chroma.upsert({
  indexName: 'test_collection',
  vectors: embeddings,
  metadata: chunks.map(chunk => ({ text: chunk.text })),
  documents: chunks.map(chunk => chunk.text),
});

View Example on GitHub

he AstraVector class provides methods to create collections and insert embeddings into DataStax Astra DB, a cloud-native vector database.

import { openai } from '@ai-sdk/openai';
import { AstraVector } from '@mastra/astra';
import { MDocument } from '@mastra/rag';
import { embedMany } from 'ai';
 
const doc = MDocument.fromText('Your text content...');
 
const chunks = await doc.chunk();
 
const { embeddings } = await embedMany({
  model: openai.embedding('text-embedding-3-small'),
  values: chunks.map(chunk => chunk.text),
});
 
const astra = new AstraVector({
  token: process.env.ASTRA_DB_TOKEN,
  endpoint: process.env.ASTRA_DB_ENDPOINT,
  keyspace: process.env.ASTRA_DB_KEYSPACE,
});
 
await astra.createIndex({
  indexName: 'test_collection',
  dimension: 1536,
});
 
await astra.upsert({
  indexName: 'test_collection',
  vectors: embeddings,
  metadata: chunks?.map(chunk => ({ text: chunk.text })),
});

The LibSQLVector class provides methods to create collections and insert embeddings into LibSQL, a fork of SQLite with vector extensions.

import { openai } from "@ai-sdk/openai";
import { LibSQLVector } from "@mastra/core/vector/libsql";
import { MDocument } from "@mastra/rag";
import { embedMany } from "ai";
 
const doc = MDocument.fromText("Your text content...");
 
const chunks = await doc.chunk();
 
const { embeddings } = await embedMany({
  values: chunks.map((chunk) => chunk.text),
  model: openai.embedding("text-embedding-3-small"),
});
 
const libsql = new LibSQLVector({
  connectionUrl: process.env.DATABASE_URL,
  authToken: process.env.DATABASE_AUTH_TOKEN, // Optional: for Turso cloud databases
});
 
await libsql.createIndex({
  indexName: "test_collection",
  dimension: 1536,
});
 
await libsql.upsert({
  indexName: "test_collection",
  vectors: embeddings,
  metadata: chunks?.map((chunk) => ({ text: chunk.text })),
});

View Example on GitHub

The UpstashVector class provides methods to create collections and insert embeddings into Upstash Vector, a serverless vector database.

import { openai } from '@ai-sdk/openai';
import { UpstashVector } from '@mastra/upstash';
import { MDocument } from '@mastra/rag';
import { embedMany } from 'ai';
 
const doc = MDocument.fromText('Your text content...');
 
const chunks = await doc.chunk();
 
const { embeddings } = await embedMany({
  values: chunks.map(chunk => chunk.text),
  model: openai.embedding('text-embedding-3-small'),
});
 
const upstash = new UpstashVector({
  url: process.env.UPSTASH_URL,
  token: process.env.UPSTASH_TOKEN,
});
 
await upstash.createIndex({
  indexName: 'test_collection',
  dimension: 1536,
});
 
await upstash.upsert({
  indexName: 'test_collection',
  vectors: embeddings,
  metadata: chunks?.map(chunk => ({ text: chunk.text })),
});

The CloudflareVector class provides methods to create collections and insert embeddings into Cloudflare Vectorize, a serverless vector database service.

import { openai } from '@ai-sdk/openai';
import { CloudflareVector } from '@mastra/vectorize';
import { MDocument } from '@mastra/rag';
import { embedMany } from 'ai';
 
const doc = MDocument.fromText('Your text content...');
 
const chunks = await doc.chunk();
 
const { embeddings } = await embedMany({
  values: chunks.map(chunk => chunk.text),
  model: openai.embedding('text-embedding-3-small'),
});
 
const vectorize = new CloudflareVector({
  accountId: process.env.CF_ACCOUNT_ID,
  apiToken: process.env.CF_API_TOKEN,
});
 
await vectorize.createIndex({
  indexName: 'test_collection',
  dimension: 1536,
});
 
await vectorize.upsert({
  indexName: 'test_collection',
  vectors: embeddings,
  metadata: chunks?.map(chunk => ({ text: chunk.text })),
});

Last updated on March 19, 2025

Metadata Extraction Hybrid Vector Search