Confident AI exporter

Confident AI is an LLM observability and evaluation platform for teams to build reliable AI applications in both development and production.

The @mastra/deepeval package sends your Mastra traces to Confident AI, where you can run metrics against them and track quality over time. It builds on DeepEval, the open-source evaluation SDK behind the platform.

Installation

npm

npm install @mastra/deepeval@latest

pnpm

pnpm add @mastra/deepeval@latest

Yarn

yarn add @mastra/deepeval@latest

Bun

bun add @mastra/deepeval@latest

Configuration

Prerequisites

1. Confident AI account: Sign up at confident-ai.com
2. API key: Generate one in your Confident AI project settings
3. Environment variables: Set your credentials:

.env

CONFIDENT_API_KEY=confident_proj_xxxxxxxxxxxxx

# Optional
CONFIDENT_TRACE_ENVIRONMENT=production  # Defaults to "development"

Zero-Config Setup

With environment variables set, use the exporter with no configuration:

src/mastra/index.ts

import { Mastra } from '@mastra/core'
import { Observability } from '@mastra/observability'
import { DeepEvalExporter } from '@mastra/deepeval'

export const mastra = new Mastra({
  observability: new Observability({
    configs: {
      deepeval: {
        serviceName: 'my-service',
        exporters: [new DeepEvalExporter()],
      },
    },
  }),
})

Explicit Configuration

You can also pass credentials directly (takes precedence over environment variables):

src/mastra/index.ts

import { Mastra } from '@mastra/core'
import { Observability } from '@mastra/observability'
import { DeepEvalExporter } from '@mastra/deepeval'

export const mastra = new Mastra({
  observability: new Observability({
    configs: {
      deepeval: {
        serviceName: 'my-service',
        exporters: [
          new DeepEvalExporter({
            apiKey: process.env.CONFIDENT_API_KEY,
            environment: 'production',
          }),
        ],
      },
    },
  }),
})

Metric collections

Confident AI evaluates incoming traces against metric collections defined in your project. Attach them at the trace level or per span type. Trace-level metrics run against the whole trace, and per-type metrics run against matching spans.

src/mastra/index.ts

new DeepEvalExporter({
  metricCollection: 'trace-metrics', // trace-level
  llmMetricCollection: 'llm-metrics', // applied to LLM spans
  agentMetricCollection: 'agent-metrics', // applied to agent spans
  toolMetricCollectionMap: {
    search: 'search-tool-metrics', // applied to the "search" tool
  },
})

Complete Configuration

new DeepEvalExporter({
  apiKey: process.env.CONFIDENT_API_KEY,
  environment: 'production', // Default: "development"
  name: 'my-trace', // Default: the Mastra serviceName
  tags: ['production'],
  metadata: { team: 'growth' },
})

Span type mapping

Mastra spans map to the span types shown in Confident AI:

Mastra span type	Confident AI span type
AGENT_RUN, WORKFLOW_RUN	AGENT
MODEL_GENERATION	LLM
TOOL_CALL, MCP_TOOL_CALL, PROVIDER_TOOL_CALL, CLIENT_TOOL_CALL	TOOL
RAG_EMBEDDING, RAG_VECTOR_OPERATION	RETRIEVER
All other exported span types	CUSTOM

Related

• Tracing Overview
• DeepEvalExporter reference
• Confident AI documentation