# Datadog exporter

[Datadog](https://datadoghq.com/) is a comprehensive monitoring platform with dedicated LLM Observability features. The Datadog exporter sends your traces to Datadog's LLM Observability product, providing insights into model performance, token usage, and conversation flows.

> **Also using dd-trace APM?:** If you also use `dd-trace` APM auto-instrumentation, consider the [Datadog Bridge](https://mastra.ai/docs/observability/integrations/bridges/datadog) instead. The bridge creates `dd-trace` spans in real time so HTTP and database calls inside tools and processors are correctly nested under their parent Mastra span. The exporter on its own sends LLM Observability data after execution completes, which means auto-instrumented APM spans fall back to the request handler.

## Installation

**npm**:

```bash
npm install @mastra/datadog@latest
```

**pnpm**:

```bash
pnpm add @mastra/datadog@latest
```

**Yarn**:

```bash
yarn add @mastra/datadog@latest
```

**Bun**:

```bash
bun add @mastra/datadog@latest
```

## Configuration

### Prerequisites

1. **Datadog Account**: Sign up at [datadoghq.com](https://datadoghq.com/) with LLM Observability enabled
2. **API Key**: Get your API key from Datadog Organization Settings → API Keys
3. **Environment Variables**: Set your credentials

```bash
DD_API_KEY=your-datadog-api-key
DD_LLMOBS_ML_APP=my-llm-app
DD_SITE=datadoghq.com  # Optional: defaults to datadoghq.com
DD_ENV=production      # Optional: environment name
```

### Zero-Config Setup

With environment variables set, use the exporter with no configuration:

```typescript
import { Mastra } from '@mastra/core'
import { Observability } from '@mastra/observability'
import { DatadogExporter } from '@mastra/datadog'

export const mastra = new Mastra({
  observability: new Observability({
    configs: {
      datadog: {
        serviceName: 'my-service',
        exporters: [new DatadogExporter()],
      },
    },
  }),
})
```

### Explicit Configuration

You can also pass credentials directly (takes precedence over environment variables):

```typescript
import { Mastra } from '@mastra/core'
import { Observability } from '@mastra/observability'
import { DatadogExporter } from '@mastra/datadog'

export const mastra = new Mastra({
  observability: new Observability({
    configs: {
      datadog: {
        serviceName: 'my-service',
        exporters: [
          new DatadogExporter({
            mlApp: process.env.DD_LLMOBS_ML_APP!,
            apiKey: process.env.DD_API_KEY!,
          }),
        ],
      },
    },
  }),
})
```

## Configuration options

### Complete Configuration

```typescript
new DatadogExporter({
  // Required settings
  mlApp: process.env.DD_LLMOBS_ML_APP!, // Groups traces under this ML app name
  apiKey: process.env.DD_API_KEY!, // Required for agentless mode (default)

  // Optional settings
  site: 'datadoghq.com', // Datadog site (datadoghq.eu, us3.datadoghq.com, etc.)
  service: 'my-service', // Service name (defaults to mlApp)
  env: 'production', // Environment name
  agentless: true, // true = direct HTTPS, false = local Datadog Agent

  // Advanced settings
  integrationsEnabled: false, // Enable dd-trace auto-instrumentation

  // Diagnostic logging
  logLevel: 'info', // debug | info | warn | error
})
```

### With Local Datadog Agent

If you have a Datadog Agent running locally, you can route traces through it instead of direct HTTPS:

```typescript
new DatadogExporter({
  mlApp: process.env.DD_LLMOBS_ML_APP!,
  agentless: false, // Use local Datadog Agent
  env: 'production',
})
```

Note: When using agent mode, the API key is read from the local agent's configuration.

## Span type mapping

Mastra span types are automatically mapped to Datadog LLM Observability span kinds:

| Mastra SpanType      | Datadog Kind |
| -------------------- | ------------ |
| `AGENT_RUN`          | `agent`      |
| `MODEL_GENERATION`   | `workflow`   |
| `MODEL_STEP`         | `llm`        |
| `TOOL_CALL`          | `tool`       |
| `MCP_TOOL_CALL`      | `tool`       |
| `WORKFLOW_RUN`       | `workflow`   |
| Other workflow types | `task`       |
| `GENERIC`            | `task`       |

Other/future Mastra span types will default to 'task' when mapped unless specified.

## Application Performance Monitoring

The sections above cover Mastra's [LLM Observability](https://docs.datadoghq.com/llm_observability/) integration. To trace your Mastra HTTP server routes (request latency, error tracking, service maps), use `dd-trace` directly for Datadog Application Performance Monitoring (APM).

### Prerequisites

1. **Datadog Agent**: Install a [Datadog Agent](https://docs.datadoghq.com/agent/) on the same host or accessible via network. The agent receives traces from `dd-trace` on `localhost:8126` and forwards them to Datadog. Follow the [agent installation guide](https://docs.datadoghq.com/agent/) to set it up.

2. **dd-trace package**: Install the tracing library in your project:

   **npm**:

   ```bash
   npm install dd-trace
   ```

   **pnpm**:

   ```bash
   pnpm add dd-trace
   ```

   **Yarn**:

   ```bash
   yarn add dd-trace
   ```

   **Bun**:

   ```bash
   bun add dd-trace
   ```

> **Note:** APM traces always route through the Datadog Agent. This is different from LLM Observability, which supports agentless mode (direct HTTPS to Datadog).

### APM only

Import and initialize `dd-trace` at the top of your entry file, before any other imports:

```typescript
import tracer from 'dd-trace'

tracer.init({
  service: process.env.DD_SERVICE || 'my-mastra-app',
  env: process.env.DD_ENV || 'production',
  version: process.env.DD_VERSION,
})

import { Mastra } from '@mastra/core'

export const mastra = new Mastra({
  bundler: {
    externals: [
      'dd-trace',
      '@datadog/native-metrics',
      '@datadog/native-appsec',
      '@datadog/native-iast-taint-tracking',
      '@datadog/pprof',
    ],
  },
})
```

Set the tracer metadata environment variables:

```bash
DD_SERVICE=my-mastra-app
DD_ENV=production
DD_VERSION=1.0.0
```

`dd-trace` auto-instruments popular HTTP frameworks, including those supported by Mastra's [server adapters](https://mastra.ai/docs/server/server-adapters). Inbound requests, outbound HTTP calls, and database queries appear as APM traces in Datadog.

### APM and LLM Observability

Import and initialize `dd-trace` before creating the Mastra instance. The `DatadogExporter` detects the existing tracer and skips re-initialization, adding LLM Observability on top of your APM setup:

```typescript
import tracer from 'dd-trace'

tracer.init({
  service: process.env.DD_SERVICE || 'my-mastra-app',
  env: process.env.DD_ENV || 'production',
  version: process.env.DD_VERSION,
})

import { Mastra } from '@mastra/core'
import { Observability } from '@mastra/observability'
import { DatadogExporter } from '@mastra/datadog'

export const mastra = new Mastra({
  observability: new Observability({
    configs: {
      datadog: {
        serviceName: 'my-mastra-app',
        exporters: [
          new DatadogExporter({
            mlApp: process.env.DD_LLMOBS_ML_APP!,
            apiKey: process.env.DD_API_KEY!,
          }),
        ],
      },
    },
  }),
  bundler: {
    externals: [
      'dd-trace',
      '@datadog/native-metrics',
      '@datadog/native-appsec',
      '@datadog/native-iast-taint-tracking',
      '@datadog/pprof',
    ],
  },
})
```

```bash
DD_SERVICE=my-mastra-app
DD_ENV=production
DD_VERSION=1.0.0
DD_API_KEY=your-datadog-api-key
DD_LLMOBS_ML_APP=my-llm-app
```

Server routes appear as APM traces and LLM calls appear as LLM Observability spans, all under the same service in Datadog.

> **Note:** Import and initialize `dd-trace` before all other modules. This allows its auto-instrumentation to patch HTTP, database, and framework libraries at load time.

## Troubleshooting

### Native module ABI mismatch

If you see errors like:

```text
Error: No native build was found for runtime=node abi=137 platform=linuxglibc arch=x64
```

This indicates a Node.js version compatibility issue with `dd-trace`'s native modules. These native modules are **optional** and provide performance monitoring features.

**Solutions:**

1. **Use Node.js 22.x**: Native modules have the best compatibility with Node.js 22.x.

2. **Ignore native module warnings**: The native modules (`@datadog/native-metrics`, `@datadog/native-appsec`, etc.) are optional. If they fail to load, core tracing functionality still works.

### Bundler externals configuration

When using bundlers like esbuild, webpack, or the Mastra CLI bundler, you may need to mark `dd-trace` and its dependencies as external:

```typescript
export const mastra = new Mastra({
  bundler: {
    externals: [
      'dd-trace',
      '@datadog/native-metrics',
      '@datadog/native-appsec',
      '@datadog/native-iast-taint-tracking',
      '@datadog/pprof',
    ],
  },
})
```

## Related

- [Tracing overview](https://mastra.ai/docs/observability/tracing/overview)
- [Datadog LLM Observability documentation](https://docs.datadoghq.com/llm_observability/)
- [Datadog APM documentation](https://docs.datadoghq.com/tracing/)