Datasets

Added in: @mastra/core@1.4.0

Datasets are collections of test cases that you run experiments against to measure how well your agents and workflows perform. Each mutation creates a new version, so you can reproduce past experiments exactly. Pair datasets with scorers to track quality across prompts, models, or code changes.

For AI agents: Run npx mastra api dataset create '{"name":"translation-pairs"}' to create a dataset directly instead of opening Studio or writing a temporary script. The command requires a running Mastra server with a storage adapter that supports datasets; start the local server with npx mastra dev, or pass the reachable server's base URL with --url. Run npx mastra api dataset create --schema before adding different dataset fields. Install Mastra's skill with npx skills add mastra-ai/skills --skill mastra for complete API CLI discovery, targeting, schema, authentication, and error-handling guidance.

Usage

Configure storage

Configure storage in your Mastra instance. Datasets require a storage adapter that provides the datasets domain:

src/mastra/index.ts

import { Mastra } from '@mastra/core'
import { LibSQLStore } from '@mastra/libsql'

export const mastra = new Mastra({
  storage: new LibSQLStore({
    id: 'my-store',
    url: 'file:./mastra.db',
  }),
})

Accessing the datasets API

All dataset operations are available through mastra.datasets:

const datasets = mastra.datasets

// Create a dataset
const dataset = await datasets.create({ name: 'my-dataset' })

// Retrieve an existing dataset
const existing = await datasets.get({ id: 'dataset-id' })

// List all datasets
const { datasets: all } = await datasets.list()

Visit the DatasetsManager reference for the full list of methods.

Studio

You can also manage datasets in Studio. After opening Studio, select Datasets from the sidebar to see all your available datasets or create a new one.

To get started, select Create Dataset and set a name, description, and optional schemas. After confirming, you'll see the dataset details page with two tabs: Items and Experiments.

In the Items view you can add, update, and delete items, and view version history. Select Add Item to insert a new item with JSON editors for input and ground truth. From this view you can also import items in bulk from a CSV or JSON file. When importing, map each column to the corresponding dataset field.

Select Versions to see the full history of changes to the dataset. After selecting Compare Versions, choose any two versions and select Compare to see a side-by-side diff of all items that were added, changed, or removed between those versions.

Creating a dataset

Call create() with a name and optional description:

src/mastra/datasets/create-dataset.ts

import { mastra } from '../index'

const dataset = await mastra.datasets.create({
  name: 'translation-pairs',
  description: 'English to Spanish translation test cases',
})

console.log(dataset.id) // auto-generated UUID

Defining schemas

You can enforce the shape of input and groundTruth by passing a Standard JSON Schema (Zod, Valibot, ArkType, etc.) when creating the dataset:

src/mastra/datasets/create-with-schema.ts

import { z } from 'zod'
import { mastra } from '../index'

const dataset = await mastra.datasets.create({
  name: 'translation-pairs',
  inputSchema: z.object({
    text: z.string(),
    sourceLang: z.string(),
    targetLang: z.string(),
  }),
  groundTruthSchema: z.object({
    translation: z.string(),
  }),
})

Items that don't match the schema are rejected at insert time.

Adding items

Use addItem() for a single item or addItems() to insert in bulk:

src/mastra/datasets/add-items.ts

// Single item
await dataset.addItem({
  input: { text: 'Hello', sourceLang: 'en', targetLang: 'es' },
  groundTruth: { translation: 'Hola' },
})

// Bulk insert
await dataset.addItems({
  items: [
    {
      input: { text: 'Goodbye', sourceLang: 'en', targetLang: 'es' },
      groundTruth: { translation: 'Adiós' },
    },
    {
      input: { text: 'Thank you', sourceLang: 'en', targetLang: 'es' },
      groundTruth: { translation: 'Gracias' },
    },
  ],
})

Updating, deleting, and purging items

updateItem(), deleteItem(), and deleteItems() create new dataset versions as they modify or remove items:

src/mastra/datasets/update-delete.ts

await dataset.updateItem({
  itemId: 'item-abc-123',
  groundTruth: { translation: '¡Hola!' },
})

await dataset.deleteItem({ itemId: 'item-abc-123' })

await dataset.deleteItems({ itemIds: ['item-1', 'item-2'] })

Deleting an item hides it from the current dataset version but retains its content in historical rows and the deletion tombstone. Use purgeItem() to redact the item's stored content across its existing history:

src/mastra/datasets/purge-item.ts

await dataset.purgeItem({ itemId: 'item-abc-123' })

Purging replaces content in every historical row and deletion tombstone with redacted values, and scrubs linked experiment-result payloads, tags, and comments. Later experiment-result submissions for the item are also stored with redacted content.

Purge serializes or conflicts with concurrent dataset item writers without guaranteeing which operation completes first. If a mutating updateItem() call loses the race, it re-reads the purge marker and rejects with DATASET_ITEM_PURGED. deleteItem() remains idempotent, and any deletion tombstone created during the race stays redacted.

Normal item mutations use Slowly Changing Dimension Type 2 (SCD-2) versioning. Permanent purge intentionally overrides historical immutability for erasure while preserving item identity and the dataset version timeline. It doesn't create a dataset version and can't be undone. Experiment counters and review status are also preserved. Avoid storing sensitive data in externalId, which remains unchanged as the item's identity key.

MongoDB storage requires a replica set or sharded deployment with transaction support for this operation. Purging fails before changing data when MongoDB transactions aren't available.

Listing and searching items

listItems() supports pagination and full-text search:

src/mastra/datasets/list-items.ts

// Paginated list
const { items, pagination } = await dataset.listItems({
  page: 0,
  perPage: 50,
})

// Full-text search
const { items: matches } = await dataset.listItems({
  search: 'Hello',
})

// Search and pagination can be combined with a specific version
const { items: v2Matches } = await dataset.listItems({
  version: 2,
  search: 'Hello',
  page: 0,
  perPage: 50,
})

// Version-only snapshot returns a bare DatasetItem[]
const v2Items = await dataset.listItems({ version: 2 })

Versioning

Adding, updating, or deleting dataset items bumps the dataset version. Purging an item's stored content with purgeItem() doesn't create a new version. This lets you pin experiments to a specific snapshot of the data while erasing sensitive content without changing the version history.

Listing versions

Use listVersions() to see the paginated history of versions:

src/mastra/datasets/list-versions.ts

const { versions, pagination } = await dataset.listVersions()

for (const v of versions) {
  console.log(`Version ${v.version} — created ${v.createdAt}`)
}

Viewing item history

See how a specific item changed across versions by calling getItemHistory() with the itemId:

src/mastra/datasets/item-history.ts

const history = await dataset.getItemHistory({ itemId: 'item-abc-123' })

for (const row of history) {
  console.log(`Version ${row.datasetVersion}`, row.input, row.groundTruth)
}

Pinning to a version

Fetch the exact items that existed at a past version:

src/mastra/datasets/pin-version.ts

const items = await dataset.listItems({ version: 2 })

You can also pin experiments to a version, see running experiments. Visit the Dataset reference for the full list of methods and parameters.

Related

• Experiments
• Scorers overview
• DatasetsManager reference
• Dataset reference
• 📹 Mastra evals, datasets, and experiments workshop