Introducing Rubric Scorers for Mastra Agents

You can now evaluate your agent's output with a rubric scorer. Define an array of criteria and an LLM-as-judge grades responses against each one.

A rubric scorer returns a pass/fail score (1 or 0) and a reason summarizing each criterion's outcome. Failed checks are fed back to the agent so it can re-run — set strategy: 'all' if every criteria should pass, or 'any' if one is enough.

Before rubric scorers, checking an agent's output against a checklist meant using a post-run eval or hand-rolling a retry loop that fired on failure. With a rubric scorer, the judge validates against the checklist as part of the agentic loop.

Attach it declaratively to your agent's config, or imperatively at runtime when you call .stream() or .generate().

Get started

Install the latest Mastra evals package:

npm install @mastra/evals

Define a rubric with createRubricScorer — pass a judge model and an array of criteria:

import { createRubricScorer } from "@mastra/evals/scorers/prebuilt";
 
export const matchReportRubricScorer = createRubricScorer({
  model: "anthropic/claude-haiku-4-5",
  criteria: [
    { description: "A `## Result` section states the score in the format `Team A X Vs Y Team B`." },
    { description: "A `## Line-ups` section lists exactly 11 named players for each team." },
    { description: "A `## Match Report` section contains 2 short prose paragraphs." },
    { description: "The report ends with a `## Sources` section with at least 2 external `https://` links." }
  ]
});

Attach the scorer to your agent's isTaskComplete config to grade every run:

import { Agent } from "@mastra/core/agent";
import { matchReportRubricScorer } from "../scorers/match-report-rubric-scorer";
 
export const matchReporterAgent = new Agent({
  id: "match-reporter",
  name: "Match Reporter",
  instructions: /* ... */,
  model: "anthropic/claude-sonnet-5",
  defaultOptions: {
    maxSteps: 6,
    isTaskComplete: {
      scorers: [matchReportRubricScorer],
      strategy: "all"
    }
  }
});

Or attach it per call on .stream() or .generate():

const stream = await agent.stream("Write a report for England Vs France World Cup 2026", {
  maxSteps: 6,
  isTaskComplete: {
    scorers: [matchReportRubricScorer],
    strategy: "all"
  }
});
 
for await (const chunk of stream.fullStream) {
  if (chunk.type === "is-task-complete") {
    console.log("results:", chunk.payload.results);
  }
}

A failed check would output something similar to the below:

[
  {
    "score": 0,
    "passed": false,
    "reason": "❌ Rubric not yet satisfied...",
    "scorerId": "rubric-scorer",
    "scorerName": "Rubric (LLM)",
    "duration": 51993
  }
]

A passed check would output something similar to the below:

[
  {
    "score": 1,
    "passed": true,
    "reason": "✅ Rubric satisfied: every required criterion is met...",
    "scorerId": "rubric-scorer",
    "scorerName": "Rubric (LLM)",
    "duration": 36499
  }
]

For more information and full configuration options, see:

• Rubric scorer reference
• Supervisor agents — Rubric scorer
• isTaskComplete on stream()