Introducing Filesystem Search for Mastra Workspaces

You can now search stored content with filesystem search. Index files from disk, databases, or APIs. Query by keyword, semantic-similarity, or both.

Knowledge is stored everywhere, usually across different and unconnected services. Mastra workspaces let you unify data sources and create searchable indexes that agents can use to find relevant information and answer user queries more accurately.

Before filesystem search, finding content meant reading every file, running grep for literal matches, or standing up a RAG pipeline. Now, RAG is part of the workspace setup and can work alongside BM25 keyword search.

BM25 is a keyword-ranking algorithm that scores documents by term frequency and length. Indexes are stored in-memory and don't require external network calls.

Vector search is a semantic-similarity approach that matches documents by their content. Indexes are stored in an external database and require network calls to embed and retrieve results.

Get started

Install the packages for the vector store and embedder you want to use. This example uses Postgres and OpenAI:

Terminal

npm install @mastra/pg @ai-sdk/openai ai

note

Requires @mastra/core@1.31.0 or later, added in PR #14735.

Create a workspace to manage the search mode, file locations, and vector store setup:

The filesystem can be a LocalFilesystem, a remote provider like S3 or Google Drive, or a combination of the two.

autoIndexPaths reads matching files at startup:

With bm25: true, each file is tokenized into an in-memory inverted index.
With vectorStore and embedder configured, each file is chunked, embedded, and stored as a row in a Postgres vector store table named search_workspace. Workspace tools include read, write, list, and delete functionality that attached agents can use.

src/mastra/workspaces/search-workspace.ts

import { LocalFilesystem, WORKSPACE_TOOLS, Workspace } from "@mastra/core/workspace";
import { PgVector } from "@mastra/pg";
import { openai } from "@ai-sdk/openai";
import { embedMany } from "ai";
 
export const searchWorkspace = new Workspace({
  id: "search-workspace",
  name: "Search Workspace",
  bm25: true,
  filesystem: new LocalFilesystem({
    basePath: "local-fs"
  }),
  autoIndexPaths: ["docs/**/*.md", "changelog/**/*.md"],
  embedder: Object.assign(
    async (texts: string[]) => {
      const { embeddings } = await embedMany({
        model: openai.embedding("text-embedding-3-small"),
        values: texts
      });
      return embeddings;
    },
    { batch: true as const, maxBatchSize: 2048 }
  ),
  searchIndexName: "search_workspace",
  vectorStore: new PgVector({
    id: "search-workspace-vectors",
    connectionString: process.env.DATABASE_URL!
  }),
  tools: {
    [WORKSPACE_TOOLS.FILESYSTEM.READ_FILE]: { enabled: true }
  }
});

Register the workspace on your main Mastra instance and call .init() to create indexes for files matched by autoIndexPaths:

src/mastra/index.ts

import { Mastra } from "@mastra/core/mastra";
import { searchWorkspace } from "./workspaces/search-workspace";
 
export const mastra = new Mastra({
  // ...
  workspace: searchWorkspace
});
 
await searchWorkspace.init();

Custom tools can call .search() to query in-memory and vector store indexes and pass the results back to the agent:

src/mastra/tools/search-tool.ts

import { createTool } from "@mastra/core/tools";
import { z } from "zod";
import { searchWorkspace } from "../workspaces/search-workspace";
 
export const searchTool = createTool({
  id: "search",
  description: "Search the workspace index for content matching the query.",
  inputSchema: z.object({
    query: z.string(),
    topK: z.number().optional().default(5),
    mode: z.enum(["bm25", "vector", "hybrid"]).optional()
  }),
  execute: async ({ query, topK, mode }) => {
    return await searchWorkspace.search(query, { topK, mode });
  }
});

Use the agent's instructions to tell it which search mode to use, then attach the workspace and tool:

src/mastra/agents/search-agent.ts

import { Agent } from "@mastra/core/agent";
import { searchWorkspace } from "../workspaces/search-workspace";
import { searchTool } from "../tools/search-tool";
 
export const searchAgent = new Agent({
  id: "search-agent",
  name: "Search Agent",
  instructions: `Answer questions by searching the workspace.
- Use 'bm25' for exact class or feature names.
- Use 'vector' for natural-language questions.
- Use 'hybrid' when the query mixes both.`,
  model: "openai/gpt-6-astra",
  workspace: searchWorkspace,
  tools: { searchTool }
});

Querying indexes

You can query the vector store directly using SQL by passing a vector_id:

SELECT vector_id, metadata, LEFT(metadata->>'text', 500) AS excerpt
FROM search_workspace
WHERE vector_id = 'docs/reference/mastra-factory-api.md#chunk-0';

Or query all stored rows:

SELECT vector_id, LEFT(metadata->>'text', 200) AS excerpt
FROM search_workspace
ORDER BY vector_id;

You can also query indexes directly using .search() with a query and options:

const query = "Factory";
 
await searchWorkspace.search(query, {
  mode: "bm25",
  topK: 5
});

For more information and full configuration options, see: