Skip to main content

dataset.runExperimentItem()

Added in: @mastra/core@1.61.0

Executes one experiment item server-side: runs the experiment's target against the item, runs the resolved scorers, and upserts the result row keyed by (experimentId, itemId, attempt).

Built for caller-driven loops where a durable orchestrator (for example a Temporal workflow) fans out one call per item and owns retries and timeouts. A retried call converges on the same row. Requires an experiment created with a target via createExperiment().

Scorers resolve with the same precedence as native runs: experiment scorers win over item scorerIds, which win over dataset scorerIds.

Usage example
Direct link to Usage example

const { experimentId } = await dataset.createExperiment({
  targetType: 'agent',
  targetId: 'translation-agent',
  scorers: ['accuracy'],
})

const { result, scores } = await dataset.runExperimentItem({
  experimentId,
  itemId: 'item-1',
})

console.log(result.output)
console.log(scores)

Each call executes the item exactly once, with no internal retry loop. Calling it on a target-less experiment throws EXPERIMENT_HAS_NO_TARGET. Use submitExperimentResult() for those. Calls after finalization throw EXPERIMENT_ALREADY_FINALIZED, and an itemId that isn't visible at the pinned dataset version throws DATASET_ITEM_NOT_FOUND.

Parameters
Direct link to Parameters

experimentId:

string
ID of an experiment created with a target.

itemId:

string
ID of the dataset item to execute. Must be visible at the pinned dataset version.

attempt?:

number
Attempt number for repeated trials. Defaults to 0. Same (experimentId, itemId, attempt) upserts the existing row.

requestContext?:

Record<string, unknown>
Request context merged with the item's own request context (item values win).

Returns
Direct link to Returns

result:

Promise<object>
The persisted result and its scores.
object

result:

ExperimentResult
The persisted result row, including output, error, traceId, attempt, and timestamps.

scores:

ScorerResult[]
Scores produced by the resolved scorers for this item.
On this page