Skip to Content
ReferenceScorersexp.MastraScorer Class

MastraScorer

The MastraScorer class is the base class for all scorers in Mastra. It provides a standard .run() method for evaluating input/output pairs and supports multi-step scoring workflows with preprocess → analyze → generateScore → generateReason execution flow.

Note: Most users should use createScorer to create scorer instances. Direct instantiation of MastraScorer is not recommended.

How to Get a MastraScorer Instance

Use the createScorer factory function, which returns a MastraScorer instance:

const scorer = createScorer({
  name: "My Custom Scorer",
  description: "Evaluates responses based on custom criteria"
})
.generateScore(({ run, results }) => {
  // scoring logic
  return 0.85;
});
 
// scorer is now a MastraScorer instance

.run() Method

The .run() method is the primary way to execute your scorer and evaluate input/output pairs. It processes the data through your defined steps (preprocess → analyze → generateScore → generateReason) and returns a comprehensive result object with the score, reasoning, and intermediate results.

const result = await scorer.run({
  input: "What is machine learning?",
  output: "Machine learning is a subset of artificial intelligence...",
  runId: "optional-run-id",
  runtimeContext: { /* optional context */ }
});

.run() Input

input:

any
Input data to be evaluated. Can be any type depending on your scorer's requirements.

output:

any
Output data to be evaluated. Can be any type depending on your scorer's requirements.

runId:

string
Optional unique identifier for this scoring run.

runtimeContext:

any
Optional runtime context from the agent or workflow step being evaluated.

.run() Returns

runId:

string
The unique identifier for this scoring run.

score:

number
Numerical score computed by the generateScore step.

reason:

string
Explanation for the score, if generateReason step was defined (optional).

preprocessStepResult:

any
Result of the preprocess step, if defined (optional).

analyzeStepResult:

any
Result of the analyze step, if defined (optional).

preprocessPrompt:

string
Preprocess prompt, if defined (optional).

analyzePrompt:

string
Analyze prompt, if defined (optional).

generateScorePrompt:

string
Generate score prompt, if defined (optional).

generateReasonPrompt:

string
Generate reason prompt, if defined (optional).

Step Execution Flow

When you call .run(), the MastraScorer executes the defined steps in this order:

  1. preprocess (optional) - Extracts or transforms data
  2. analyze (optional) - Processes the input/output and preprocessed data
  3. generateScore (required) - Computes the numerical score
  4. generateReason (optional) - Provides explanation for the score

Each step receives the results from previous steps, allowing you to build complex evaluation pipelines.

Usage Example

const scorer = createScorer({
  name: "Quality Scorer",
  description: "Evaluates response quality"
})
.preprocess(({ run }) => {
  // Extract key information
  return { wordCount: run.output.split(' ').length };
})
.analyze(({ run, results }) => {
  // Analyze the response
  const hasSubstance = results.preprocessStepResult.wordCount > 10;
  return { hasSubstance };
})
.generateScore(({ results }) => {
  // Calculate score
  return results.analyzeStepResult.hasSubstance ? 1.0 : 0.0;
})
.generateReason(({ score, results }) => {
  // Explain the score
  const wordCount = results.preprocessStepResult.wordCount;
  return `Score: ${score}. Response has ${wordCount} words.`;
});
 
// Use the scorer
const result = await scorer.run({
  input: "What is machine learning?",
  output: "Machine learning is a subset of artificial intelligence..."
});
 
console.log(result.score);  // 1.0
console.log(result.reason); // "Score: 1.0. Response has 12 words."

Integration

MastraScorer instances can be used for agents and workflow steps

See the createScorer reference for detailed information on defining custom scoring logic.

ToxicitycreateScorer