Skip to Content
ReferenceScorersMastraScorer Class

MastraScorer

The MastraScorer class is the base class for all scorers in Mastra. It provides a standard .run() method for evaluating input/output pairs and supports multi-step scoring workflows. Most users will use createScorer or createLLMScorer, but advanced users can subclass or instantiate MastraScorer directly for full control.

Constructor Options

name:

string
Name of the scorer.

description:

string
Description of what the scorer does.

extract:

function
Optional extraction step. See extract step signature below.

analyze:

function
Main scoring logic. See analyze step signature below.

reason:

function
Optional reason/explanation step. See reason step signature below.

metadata:

object
Optional metadata for the scorer.

isLLMScorer:

boolean
(Internal) Used to distinguish LLM scorers.

Step Function Signatures

extract

input:

Record<string, any>[]
Input records provided to the scorer. If the scorer is added to an agent, this will be an array of user messages, e.g. `[{ role: 'user', content: 'hello world' }]`. If the scorer is used in a workflow, this will be the input of the workflow.

output:

Record<string, any>
Output record provided to the scorer. For agents, this is usually the agent's response. For workflows, this is the workflow's output.

Returns: { results: any }
The method must return an object with a results property. The value of results will be passed to the analyze function as extractStepResult.

analyze

input:

Record<string, any>[]
Input records provided to the scorer. If the scorer is added to an agent, this will be an array of user messages, e.g. `[{ role: 'user', content: 'hello world' }]`. If the scorer is used in a workflow, this will be the input of the workflow.

output:

Record<string, any>
Output record provided to the scorer. For agents, this is usually the agent's response. For workflows, this is the workflow's output.

extractStepResult:

object
Result of the extract step, if defined (optional).

Returns: { score: number, results?: any }
The method must return an object with a score property (required). Optionally, it may return a results property. The value of results will be passed to the reason function as analyzeStepResult.

reason

input:

Record<string, any>[]
Input records provided to the scorer. If the scorer is added to an agent, this will be an array of user messages, e.g. `[{ role: 'user', content: 'hello world' }]`. If the scorer is used in a workflow, this will be the input of the workflow.

output:

Record<string, any>
Output record provided to the scorer. For agents, this is usually the agent's response. For workflows, this is the workflow's output.

score:

number
Score computed by the analyze step.

analyzeStepResult:

object
Result of the analyze step.

extractStepResult:

object
Result of the extract step, if defined (optional).

Returns: { reason: string }
The method must return an object with a reason property, which should be a string explaining the score.

All step functions can be async.

.run() Input

runId:

string
The id of the run (optional).

input:

Record<string, any>[]
An array of records. This should contain user messages or the data to be evaluated.

output:

Record<string, any>
A record. This should contain the output to be evaluated.

additionalContext:

Record<string, any>
Additional context for the run (optional).

runtimeContext:

Record<string, any>
Runtime context for the run (optional).

.run() Returns

runId:

string
The id of the run (optional).

extractStepResult:

object
Result of the extract step, if defined (optional).

analyzeStepResult:

object
Result of the analyze step (custom structure defined by your scorer).

score:

number
Score computed by your analyze function.

reason:

string
Reason/explanation for the score, if defined (optional).

Integration

MastraScorer instances can be used for agents and workflow steps

ToxicitycreateScorer