> Discover all available pages from the documentation index: https://mastra.ai/llms.txt

# Confident AI

[Confident AI](https://www.confident-ai.com/) is an LLM observability and evaluation platform for teams to build reliable AI applications in both development and production.

The `@mastra/deepeval` package sends your Mastra traces to Confident AI, where you can run metrics against them and track quality over time. It builds on [DeepEval](https://www.confident-ai.com/docs), the open-source evaluation SDK behind the platform.

## Installation

**npm**:

```bash
npm install @mastra/deepeval@latest
```

**pnpm**:

```bash
pnpm add @mastra/deepeval@latest
```

**Yarn**:

```bash
yarn add @mastra/deepeval@latest
```

**Bun**:

```bash
bun add @mastra/deepeval@latest
```

## Configuration

### Prerequisites

1. **Confident AI account**: Sign up at [confident-ai.com](https://www.confident-ai.com/)
2. **API key**: Generate one in your Confident AI project settings
3. **Environment variables**: Set your credentials:

```bash
CONFIDENT_API_KEY=confident_proj_xxxxxxxxxxxxx

# Optional
CONFIDENT_TRACE_ENVIRONMENT=production  # Defaults to "development"
```

### Zero-Config Setup

With environment variables set, use the exporter with no configuration:

```typescript
import { Mastra } from '@mastra/core'
import { Observability } from '@mastra/observability'
import { DeepEvalExporter } from '@mastra/deepeval'

export const mastra = new Mastra({
  observability: new Observability({
    configs: {
      deepeval: {
        serviceName: 'my-service',
        exporters: [new DeepEvalExporter()],
      },
    },
  }),
})
```

### Explicit Configuration

You can also pass credentials directly (takes precedence over environment variables):

```typescript
import { Mastra } from '@mastra/core'
import { Observability } from '@mastra/observability'
import { DeepEvalExporter } from '@mastra/deepeval'

export const mastra = new Mastra({
  observability: new Observability({
    configs: {
      deepeval: {
        serviceName: 'my-service',
        exporters: [
          new DeepEvalExporter({
            apiKey: process.env.CONFIDENT_API_KEY,
            environment: 'production',
          }),
        ],
      },
    },
  }),
})
```

### Metric collections

Confident AI evaluates incoming traces against metric collections defined in your project. Attach them at the trace level or per span type. Trace-level metrics run against the whole trace, and per-type metrics run against matching spans.

```typescript
new DeepEvalExporter({
  metricCollection: 'trace-metrics', // trace-level
  llmMetricCollection: 'llm-metrics', // applied to LLM spans
  agentMetricCollection: 'agent-metrics', // applied to agent spans
  toolMetricCollectionMap: {
    search: 'search-tool-metrics', // applied to the "search" tool
  },
})
```

### Complete Configuration

```typescript
new DeepEvalExporter({
  apiKey: process.env.CONFIDENT_API_KEY,
  environment: 'production', // Default: "development"
  name: 'my-trace', // Default: the Mastra serviceName
  tags: ['production'],
  metadata: { team: 'growth' },
})
```

## Span type mapping

Mastra spans map to the span types shown in Confident AI:

| Mastra span type                                                       | Confident AI span type |
| ---------------------------------------------------------------------- | ---------------------- |
| `AGENT_RUN`, `WORKFLOW_RUN`                                            | `AGENT`                |
| `MODEL_GENERATION`                                                     | `LLM`                  |
| `TOOL_CALL`, `MCP_TOOL_CALL`, `PROVIDER_TOOL_CALL`, `CLIENT_TOOL_CALL` | `TOOL`                 |
| `RAG_EMBEDDING`, `RAG_VECTOR_OPERATION`                                | `RETRIEVER`            |
| All other exported span types                                          | `CUSTOM`               |

## Related

- [Tracing Overview](https://mastra.ai/docs/observability/tracing/overview)
- [DeepEvalExporter reference](https://mastra.ai/reference/observability/tracing/exporters/confident-ai)
- [Confident AI documentation](https://www.confident-ai.com/docs)