Upsert Embeddings

埋め込みを生成した後、ベクトル類似性検索をサポートするデータベースに保存する必要があります。この例では、後で取得するために様々なベクトルデータベースに埋め込みを保存する方法を示します。

PgVector

PgVector クラスは、pgvector 拡張機能を利用する PostgreSQL で、インデックスの作成や埋め込みの挿入を行うためのメソッドを提供します。


import { openai } from "@ai-sdk/openai";
import { PgVector } from "@mastra/pg";
import { MDocument } from "@mastra/rag";
import { embedMany } from "ai";
 
const doc = MDocument.fromText("Your text content...");
 
const chunks = await doc.chunk();
 
const { embeddings } = await embedMany({
  values: chunks.map(chunk => chunk.text),
  model: openai.embedding("text-embedding-3-small"),
});
 
const pgVector = new PgVector({ connectionString: process.env.POSTGRES_CONNECTION_STRING! });
 
await pgVector.createIndex({
  indexName: "test_index",
  dimension: 1536,
});
 
await pgVector.upsert({
  indexName: "test_index",
  vectors: embeddings,
  metadata: chunks?.map((chunk: any) => ({ text: chunk.text })),
});

View Example on GitHub

Pinecone

PineconeVector クラスは、マネージド型ベクターデータベースサービスである Pinecone で、インデックスの作成や埋め込みの挿入を行うためのメソッドを提供します。


import { openai } from '@ai-sdk/openai';
import { PineconeVector } from '@mastra/pinecone';
import { MDocument } from '@mastra/rag';
import { embedMany } from 'ai';
 
const doc = MDocument.fromText('Your text content...');
 
const chunks = await doc.chunk();
 
const { embeddings } = await embedMany({
  values: chunks.map(chunk => chunk.text),
  model: openai.embedding('text-embedding-3-small'),
});
 
const pinecone = new PineconeVector({
  apiKey: process.env.PINECONE_API_KEY!,
});
 
await pinecone.createIndex({
  indexName: 'testindex',
  dimension: 1536,
});
 
await pinecone.upsert({
  indexName: 'testindex',
  vectors: embeddings,
  metadata: chunks?.map(chunk => ({ text: chunk.text })),
});

View Example on GitHub

Qdrant

QdrantVector クラスは、高性能なベクターデータベースである Qdrant で、コレクションの作成や埋め込みの挿入を行うためのメソッドを提供します。


import { openai } from '@ai-sdk/openai';
import { QdrantVector } from '@mastra/qdrant';
import { MDocument } from '@mastra/rag';
import { embedMany } from 'ai';
 
const doc = MDocument.fromText('Your text content...');
 
const chunks = await doc.chunk();
 
const { embeddings } = await embedMany({
  values: chunks.map(chunk => chunk.text),
  model: openai.embedding('text-embedding-3-small'),
  maxRetries: 3,
});
 
const qdrant = new QdrantVector({
  url: process.env.QDRANT_URL,
  apiKey: process.env.QDRANT_API_KEY,
});
 
await qdrant.createIndex({
  indexName: 'test_collection',
  dimension: 1536,
});
 
await qdrant.upsert({
  indexName: 'test_collection',
  vectors: embeddings,
  metadata: chunks?.map(chunk => ({ text: chunk.text })),
});

Chroma

ChromaVector クラスは、オープンソースの埋め込みデータベースである Chroma に対して、コレクションの作成や埋め込みの挿入を行うためのメソッドを提供します。


import { openai } from '@ai-sdk/openai';
import { ChromaVector } from '@mastra/chroma';
import { MDocument } from '@mastra/rag';
import { embedMany } from 'ai';
 
const doc = MDocument.fromText('Your text content...');
 
const chunks = await doc.chunk();
 
const { embeddings } = await embedMany({
  values: chunks.map(chunk => chunk.text),
  model: openai.embedding('text-embedding-3-small'),
});
 
// Running Chroma locally
const store = new ChromaVector();
 
// Running on Chroma Cloud
const store = new ChromaVector({
  apiKey: process.env.CHROMA_API_KEY,
  tenant: process.env.CHROMA_TENANT,
  database: process.env.CHROMA_DATABASE
});
 
await store.createIndex({
  indexName: 'test_collection',
  dimension: 1536,
});
 
await chroma.upsert({
  indexName: 'test_collection',
  vectors: embeddings,
  metadata: chunks.map(chunk => ({ text: chunk.text })),
  documents: chunks.map(chunk => chunk.text),
});

View Example on GitHub

Astra DB

AstraVector クラスは、クラウドネイティブなベクターデータベースである DataStax Astra DB に対して、コレクションの作成や埋め込みの挿入を行うためのメソッドを提供します。


import { openai } from '@ai-sdk/openai';
import { AstraVector } from '@mastra/astra';
import { MDocument } from '@mastra/rag';
import { embedMany } from 'ai';
 
const doc = MDocument.fromText('Your text content...');
 
const chunks = await doc.chunk();
 
const { embeddings } = await embedMany({
  model: openai.embedding('text-embedding-3-small'),
  values: chunks.map(chunk => chunk.text),
});
 
const astra = new AstraVector({
  token: process.env.ASTRA_DB_TOKEN,
  endpoint: process.env.ASTRA_DB_ENDPOINT,
  keyspace: process.env.ASTRA_DB_KEYSPACE,
});
 
await astra.createIndex({
  indexName: 'test_collection',
  dimension: 1536,
});
 
await astra.upsert({
  indexName: 'test_collection',
  vectors: embeddings,
  metadata: chunks?.map(chunk => ({ text: chunk.text })),
});

LibSQL

LibSQLVector クラスは、ベクター拡張を備えた SQLite のフォークである LibSQL に対して、コレクションの作成や埋め込みの挿入を行うためのメソッドを提供します。


import { openai } from "@ai-sdk/openai";
import { LibSQLVector } from "@mastra/core/vector/libsql";
import { MDocument } from "@mastra/rag";
import { embedMany } from "ai";
 
const doc = MDocument.fromText("Your text content...");
 
const chunks = await doc.chunk();
 
const { embeddings } = await embedMany({
  values: chunks.map((chunk) => chunk.text),
  model: openai.embedding("text-embedding-3-small"),
});
 
const libsql = new LibSQLVector({
  connectionUrl: process.env.DATABASE_URL,
  authToken: process.env.DATABASE_AUTH_TOKEN, // Optional: for Turso cloud databases
});
 
await libsql.createIndex({
  indexName: "test_collection",
  dimension: 1536,
});
 
await libsql.upsert({
  indexName: "test_collection",
  vectors: embeddings,
  metadata: chunks?.map((chunk) => ({ text: chunk.text })),
});

View Example on GitHub

Upstash

UpstashVector クラスは、コレクションの作成や、サーバーレスのベクターデータベースである Upstash Vector への埋め込みの挿入を行うためのメソッドを提供します。


import { openai } from '@ai-sdk/openai';
import { UpstashVector } from '@mastra/upstash';
import { MDocument } from '@mastra/rag';
import { embedMany } from 'ai';
 
const doc = MDocument.fromText('Your text content...');
 
const chunks = await doc.chunk();
 
const { embeddings } = await embedMany({
  values: chunks.map(chunk => chunk.text),
  model: openai.embedding('text-embedding-3-small'),
});
 
const upstash = new UpstashVector({
  url: process.env.UPSTASH_URL,
  token: process.env.UPSTASH_TOKEN,
});
 
// ここでは store.createIndex の呼び出しは不要です。Upstash では、アップサート時に
// その名前空間（Upstash では namespace と呼びます）が存在しない場合、自動的にインデックスを作成します。
await upstash.upsert({
  indexName: 'test_collection', // Upstash における namespace 名
  vectors: embeddings,
  metadata: chunks?.map(chunk => ({ text: chunk.text })),
});

Cloudflare

CloudflareVector クラスは、コレクションの作成や、サーバーレスのベクターデータベースサービスである Cloudflare Vectorize への埋め込みの挿入を行うためのメソッドを提供します。


import { openai } from '@ai-sdk/openai';
import { CloudflareVector } from '@mastra/vectorize';
import { MDocument } from '@mastra/rag';
import { embedMany } from 'ai';
 
const doc = MDocument.fromText('Your text content...');
 
const chunks = await doc.chunk();
 
const { embeddings } = await embedMany({
  values: chunks.map(chunk => chunk.text),
  model: openai.embedding('text-embedding-3-small'),
});
 
const vectorize = new CloudflareVector({
  accountId: process.env.CF_ACCOUNT_ID,
  apiToken: process.env.CF_API_TOKEN,
});
 
await vectorize.createIndex({
  indexName: 'test_collection',
  dimension: 1536,
});
 
await vectorize.upsert({
  indexName: 'test_collection',
  vectors: embeddings,
  metadata: chunks?.map(chunk => ({ text: chunk.text })),
});

MongoDB

MongoDBVector クラスは、Atlas Search を用いた MongoDB へのインデックス作成と埋め込みの挿入を行うためのメソッドを提供します。


import { openai } from "@ai-sdk/openai";
import { MongoDBVector } from "@mastra/mongodb";
import { MDocument } from "@mastra/rag";
import { embedMany } from "ai";
 
const doc = MDocument.fromText("Your text content...");
 
const chunks = await doc.chunk();
 
const { embeddings } = await embedMany({
  values: chunks.map(chunk => chunk.text),
  model: openai.embedding("text-embedding-3-small"),
});
 
const vectorDB = new MongoDBVector({
  uri: process.env.MONGODB_URI!,
  dbName: process.env.MONGODB_DB_NAME!,
});
 
await vectorDB.createIndex({
  indexName: "test_index",
  dimension: 1536,
});
 
await vectorDB.upsert({
  indexName: "test_index",
  vectors: embeddings,
  metadata: chunks?.map((chunk: any) => ({ text: chunk.text })),
});

OpenSearch

OpenSearchVector クラスは、ベクター検索機能を備えた分散型検索エンジンである OpenSearch へのインデックス作成と埋め込みの挿入を行うためのメソッドを提供します。


import { openai } from '@ai-sdk/openai';
import { OpenSearchVector } from '@mastra/opensearch';
import { MDocument } from '@mastra/rag';
import { embedMany } from 'ai';
 
const doc = MDocument.fromText('Your text content...');
 
const chunks = await doc.chunk();
 
const { embeddings } = await embedMany({
  values: chunks.map(chunk => chunk.text),
  model: openai.embedding('text-embedding-3-small'),
});
 
const vectorDB = new OpenSearchVector({
  uri: process.env.OPENSEARCH_URI!,
});
 
await vectorDB.createIndex({
  indexName: 'test_index',
  dimension: 1536,
});
 
await vectorDB.upsert({
  indexName: 'test_index',
  vectors: embeddings,
  metadata: chunks?.map((chunk: any) => ({ text: chunk.text })),
});

Couchbase

CouchbaseVector クラスは、ベクトル検索に対応した分散型 NoSQL データベースである Couchbase で、インデックスの作成やベクトル埋め込みの挿入を行うためのメソッドを提供します。


import { openai } from '@ai-sdk/openai';
import { CouchbaseVector } from '@mastra/couchbase';
import { MDocument } from '@mastra/rag';
import { embedMany } from 'ai';
 
const doc = MDocument.fromText('Your text content...');
 
const chunks = await doc.chunk();
 
const { embeddings } = await embedMany({
  values: chunks.map(chunk => chunk.text),
  model: openai.embedding('text-embedding-3-small'),
});
 
const couchbase = new CouchbaseVector({
  connectionString: process.env.COUCHBASE_CONNECTION_STRING,
  username: process.env.COUCHBASE_USERNAME,
  password: process.env.COUCHBASE_PASSWORD,
  bucketName: process.env.COUCHBASE_BUCKET,
  scopeName: process.env.COUCHBASE_SCOPE,
  collectionName: process.env.COUCHBASE_COLLECTION,
});
 
await couchbase.createIndex({
  indexName: 'test_collection',
  dimension: 1536,
});
 
await couchbase.upsert({
  indexName: 'test_collection',
  vectors: embeddings,
  metadata: chunks?.map(chunk => ({ text: chunk.text })),
});

LanceDB

LanceVectorStore クラスは、Lance カラムナ形式の上に構築された組み込み型ベクトルデータベースである LanceDB で、テーブルやインデックスの作成、ベクトル埋め込みの挿入を行うためのメソッドを提供します。


import { openai } from '@ai-sdk/openai';
import { LanceVectorStore } from '@mastra/lance';
import { MDocument } from '@mastra/rag';
import { embedMany } from 'ai';
 
const doc = MDocument.fromText('Your text content...');
 
const chunks = await doc.chunk();
 
const { embeddings } = await embedMany({
  values: chunks.map(chunk => chunk.text),
  model: openai.embedding('text-embedding-3-small'),
});
 
const lance = await LanceVectorStore.create('/path/to/db');
 
// LanceDB では、まずテーブルを作成する必要があります
await lance.createIndex({
  tableName: 'myVectors',
  indexName: 'vector',
  dimension: 1536,
});
 
await lance.upsert({
  tableName: 'myVectors',
  vectors: embeddings,
  metadata: chunks?.map(chunk => ({ text: chunk.text })),
});

Upsert Embeddings

PgVector

PgVector クラスは、pgvector 拡張機能を利用する PostgreSQL で、インデックスの作成や埋め込みの挿入を行うためのメソッドを提供します。


import { openai } from "@ai-sdk/openai";
import { PgVector } from "@mastra/pg";
import { MDocument } from "@mastra/rag";
import { embedMany } from "ai";
 
const doc = MDocument.fromText("Your text content...");
 
const chunks = await doc.chunk();
 
const { embeddings } = await embedMany({
  values: chunks.map(chunk => chunk.text),
  model: openai.embedding("text-embedding-3-small"),
});
 
const pgVector = new PgVector({ connectionString: process.env.POSTGRES_CONNECTION_STRING! });
 
await pgVector.createIndex({
  indexName: "test_index",
  dimension: 1536,
});
 
await pgVector.upsert({
  indexName: "test_index",
  vectors: embeddings,
  metadata: chunks?.map((chunk: any) => ({ text: chunk.text })),
});

View Example on GitHub

Pinecone


import { openai } from '@ai-sdk/openai';
import { PineconeVector } from '@mastra/pinecone';
import { MDocument } from '@mastra/rag';
import { embedMany } from 'ai';
 
const doc = MDocument.fromText('Your text content...');
 
const chunks = await doc.chunk();
 
const { embeddings } = await embedMany({
  values: chunks.map(chunk => chunk.text),
  model: openai.embedding('text-embedding-3-small'),
});
 
const pinecone = new PineconeVector({
  apiKey: process.env.PINECONE_API_KEY!,
});
 
await pinecone.createIndex({
  indexName: 'testindex',
  dimension: 1536,
});
 
await pinecone.upsert({
  indexName: 'testindex',
  vectors: embeddings,
  metadata: chunks?.map(chunk => ({ text: chunk.text })),
});

View Example on GitHub

Qdrant

QdrantVector クラスは、高性能なベクターデータベースである Qdrant で、コレクションの作成や埋め込みの挿入を行うためのメソッドを提供します。


import { openai } from '@ai-sdk/openai';
import { QdrantVector } from '@mastra/qdrant';
import { MDocument } from '@mastra/rag';
import { embedMany } from 'ai';
 
const doc = MDocument.fromText('Your text content...');
 
const chunks = await doc.chunk();
 
const { embeddings } = await embedMany({
  values: chunks.map(chunk => chunk.text),
  model: openai.embedding('text-embedding-3-small'),
  maxRetries: 3,
});
 
const qdrant = new QdrantVector({
  url: process.env.QDRANT_URL,
  apiKey: process.env.QDRANT_API_KEY,
});
 
await qdrant.createIndex({
  indexName: 'test_collection',
  dimension: 1536,
});
 
await qdrant.upsert({
  indexName: 'test_collection',
  vectors: embeddings,
  metadata: chunks?.map(chunk => ({ text: chunk.text })),
});

Chroma


import { openai } from '@ai-sdk/openai';
import { ChromaVector } from '@mastra/chroma';
import { MDocument } from '@mastra/rag';
import { embedMany } from 'ai';
 
const doc = MDocument.fromText('Your text content...');
 
const chunks = await doc.chunk();
 
const { embeddings } = await embedMany({
  values: chunks.map(chunk => chunk.text),
  model: openai.embedding('text-embedding-3-small'),
});
 
// Running Chroma locally
const store = new ChromaVector();
 
// Running on Chroma Cloud
const store = new ChromaVector({
  apiKey: process.env.CHROMA_API_KEY,
  tenant: process.env.CHROMA_TENANT,
  database: process.env.CHROMA_DATABASE
});
 
await store.createIndex({
  indexName: 'test_collection',
  dimension: 1536,
});
 
await chroma.upsert({
  indexName: 'test_collection',
  vectors: embeddings,
  metadata: chunks.map(chunk => ({ text: chunk.text })),
  documents: chunks.map(chunk => chunk.text),
});

View Example on GitHub

Astra DB


import { openai } from '@ai-sdk/openai';
import { AstraVector } from '@mastra/astra';
import { MDocument } from '@mastra/rag';
import { embedMany } from 'ai';
 
const doc = MDocument.fromText('Your text content...');
 
const chunks = await doc.chunk();
 
const { embeddings } = await embedMany({
  model: openai.embedding('text-embedding-3-small'),
  values: chunks.map(chunk => chunk.text),
});
 
const astra = new AstraVector({
  token: process.env.ASTRA_DB_TOKEN,
  endpoint: process.env.ASTRA_DB_ENDPOINT,
  keyspace: process.env.ASTRA_DB_KEYSPACE,
});
 
await astra.createIndex({
  indexName: 'test_collection',
  dimension: 1536,
});
 
await astra.upsert({
  indexName: 'test_collection',
  vectors: embeddings,
  metadata: chunks?.map(chunk => ({ text: chunk.text })),
});

LibSQL


import { openai } from "@ai-sdk/openai";
import { LibSQLVector } from "@mastra/core/vector/libsql";
import { MDocument } from "@mastra/rag";
import { embedMany } from "ai";
 
const doc = MDocument.fromText("Your text content...");
 
const chunks = await doc.chunk();
 
const { embeddings } = await embedMany({
  values: chunks.map((chunk) => chunk.text),
  model: openai.embedding("text-embedding-3-small"),
});
 
const libsql = new LibSQLVector({
  connectionUrl: process.env.DATABASE_URL,
  authToken: process.env.DATABASE_AUTH_TOKEN, // Optional: for Turso cloud databases
});
 
await libsql.createIndex({
  indexName: "test_collection",
  dimension: 1536,
});
 
await libsql.upsert({
  indexName: "test_collection",
  vectors: embeddings,
  metadata: chunks?.map((chunk) => ({ text: chunk.text })),
});

View Example on GitHub

Upstash


import { openai } from '@ai-sdk/openai';
import { UpstashVector } from '@mastra/upstash';
import { MDocument } from '@mastra/rag';
import { embedMany } from 'ai';
 
const doc = MDocument.fromText('Your text content...');
 
const chunks = await doc.chunk();
 
const { embeddings } = await embedMany({
  values: chunks.map(chunk => chunk.text),
  model: openai.embedding('text-embedding-3-small'),
});
 
const upstash = new UpstashVector({
  url: process.env.UPSTASH_URL,
  token: process.env.UPSTASH_TOKEN,
});
 
// ここでは store.createIndex の呼び出しは不要です。Upstash では、アップサート時に
// その名前空間（Upstash では namespace と呼びます）が存在しない場合、自動的にインデックスを作成します。
await upstash.upsert({
  indexName: 'test_collection', // Upstash における namespace 名
  vectors: embeddings,
  metadata: chunks?.map(chunk => ({ text: chunk.text })),
});

Cloudflare


import { openai } from '@ai-sdk/openai';
import { CloudflareVector } from '@mastra/vectorize';
import { MDocument } from '@mastra/rag';
import { embedMany } from 'ai';
 
const doc = MDocument.fromText('Your text content...');
 
const chunks = await doc.chunk();
 
const { embeddings } = await embedMany({
  values: chunks.map(chunk => chunk.text),
  model: openai.embedding('text-embedding-3-small'),
});
 
const vectorize = new CloudflareVector({
  accountId: process.env.CF_ACCOUNT_ID,
  apiToken: process.env.CF_API_TOKEN,
});
 
await vectorize.createIndex({
  indexName: 'test_collection',
  dimension: 1536,
});
 
await vectorize.upsert({
  indexName: 'test_collection',
  vectors: embeddings,
  metadata: chunks?.map(chunk => ({ text: chunk.text })),
});

MongoDB

MongoDBVector クラスは、Atlas Search を用いた MongoDB へのインデックス作成と埋め込みの挿入を行うためのメソッドを提供します。


import { openai } from "@ai-sdk/openai";
import { MongoDBVector } from "@mastra/mongodb";
import { MDocument } from "@mastra/rag";
import { embedMany } from "ai";
 
const doc = MDocument.fromText("Your text content...");
 
const chunks = await doc.chunk();
 
const { embeddings } = await embedMany({
  values: chunks.map(chunk => chunk.text),
  model: openai.embedding("text-embedding-3-small"),
});
 
const vectorDB = new MongoDBVector({
  uri: process.env.MONGODB_URI!,
  dbName: process.env.MONGODB_DB_NAME!,
});
 
await vectorDB.createIndex({
  indexName: "test_index",
  dimension: 1536,
});
 
await vectorDB.upsert({
  indexName: "test_index",
  vectors: embeddings,
  metadata: chunks?.map((chunk: any) => ({ text: chunk.text })),
});

OpenSearch


import { openai } from '@ai-sdk/openai';
import { OpenSearchVector } from '@mastra/opensearch';
import { MDocument } from '@mastra/rag';
import { embedMany } from 'ai';
 
const doc = MDocument.fromText('Your text content...');
 
const chunks = await doc.chunk();
 
const { embeddings } = await embedMany({
  values: chunks.map(chunk => chunk.text),
  model: openai.embedding('text-embedding-3-small'),
});
 
const vectorDB = new OpenSearchVector({
  uri: process.env.OPENSEARCH_URI!,
});
 
await vectorDB.createIndex({
  indexName: 'test_index',
  dimension: 1536,
});
 
await vectorDB.upsert({
  indexName: 'test_index',
  vectors: embeddings,
  metadata: chunks?.map((chunk: any) => ({ text: chunk.text })),
});

Couchbase


import { openai } from '@ai-sdk/openai';
import { CouchbaseVector } from '@mastra/couchbase';
import { MDocument } from '@mastra/rag';
import { embedMany } from 'ai';
 
const doc = MDocument.fromText('Your text content...');
 
const chunks = await doc.chunk();
 
const { embeddings } = await embedMany({
  values: chunks.map(chunk => chunk.text),
  model: openai.embedding('text-embedding-3-small'),
});
 
const couchbase = new CouchbaseVector({
  connectionString: process.env.COUCHBASE_CONNECTION_STRING,
  username: process.env.COUCHBASE_USERNAME,
  password: process.env.COUCHBASE_PASSWORD,
  bucketName: process.env.COUCHBASE_BUCKET,
  scopeName: process.env.COUCHBASE_SCOPE,
  collectionName: process.env.COUCHBASE_COLLECTION,
});
 
await couchbase.createIndex({
  indexName: 'test_collection',
  dimension: 1536,
});
 
await couchbase.upsert({
  indexName: 'test_collection',
  vectors: embeddings,
  metadata: chunks?.map(chunk => ({ text: chunk.text })),
});

LanceDB


import { openai } from '@ai-sdk/openai';
import { LanceVectorStore } from '@mastra/lance';
import { MDocument } from '@mastra/rag';
import { embedMany } from 'ai';
 
const doc = MDocument.fromText('Your text content...');
 
const chunks = await doc.chunk();
 
const { embeddings } = await embedMany({
  values: chunks.map(chunk => chunk.text),
  model: openai.embedding('text-embedding-3-small'),
});
 
const lance = await LanceVectorStore.create('/path/to/db');
 
// LanceDB では、まずテーブルを作成する必要があります
await lance.createIndex({
  tableName: 'myVectors',
  indexName: 'vector',
  dimension: 1536,
});
 
await lance.upsert({
  tableName: 'myVectors',
  vectors: embeddings,
  metadata: chunks?.map(chunk => ({ text: chunk.text })),
});