Durable agents

Added in: @mastra/core@1.45.0

beta

Breaking changes may occur without a major version bump until the API is stable.

A durable agent wraps a regular Agent so the agentic loop runs inside a workflow. Events flow through PubSub, which means a client can disconnect and reconnect without missing chunks. The run state is persisted, so it survives process restarts.

When to use durable agents

Use a durable agent when any of the following apply:

• The client may drop and reconnect mid-stream (mobile, spotty networks, long-running calls).
• The agentic loop may outlive a single HTTP request (background research, multi-step tool use).
• You need an observe/reconnect API where a second client picks up a stream that a first client started.
• You want Inngest-powered execution with step memoization, retries, and monitoring.

For short-lived, request-scoped calls where the client stays connected, a regular Agent with stream() or generate() is simpler.

Quickstart

Wrap an existing agent with createDurableAgent() from @mastra/core/agent/durable:

src/mastra/agents/researcher.ts

import { Agent } from '@mastra/core/agent'
import { createDurableAgent } from '@mastra/core/agent/durable'

const agent = new Agent({
  id: 'researcher',
  name: 'Researcher',
  instructions: 'You research topics thoroughly.',
  model: 'openai/gpt-5.6-sol',
})

export const durableResearcher = createDurableAgent({ agent })

Register the durable agent with Mastra and call stream():

src/mastra/index.ts

import { Mastra } from '@mastra/core'
import { durableResearcher } from './agents/researcher'

const mastra = new Mastra({
  agents: { durableResearcher },
})

const { output, runId, cleanup } = await durableResearcher.stream(
  'Research quantum computing advances in 2025',
)

for await (const chunk of output.fullStream) {
  // Process each chunk as it arrives
}

// Release PubSub subscriptions and clear the run from the registry.
// If you skip this, an automatic cleanup timer fires after the stream ends.
cleanup()

The returned runId identifies the execution. Pass it to observe() to reconnect from a different client. Visit the DurableAgent reference for the full configuration and method API.

How it works

A durable agent adds three layers on top of a regular agent:

1. Workflow execution: stream() serializes the messages and options into a workflow input, then triggers the agentic loop inside a durable workflow. The workflow runs the same loop as Agent.stream() but each step can be memoized and replayed.

2. PubSub streaming: The loop publishes chunks to a PubSub topic keyed by the run ID. The caller subscribes to that topic and pipes the chunks into a ReadableStream, while the cache replays any chunks missed during a disconnect.

3. Cache layer: An optional cache (in-memory by default, Redis or another backend in production) stores published events so that a late subscriber can catch up.

Execution variants

Mastra provides three factory functions that produce durable agents. They differ in how the workflow is executed:

Factory	Package	Best for
createDurableAgent()	@mastra/core	Local development and single-process servers. You get a stream you can await directly.
createEventedAgent()	@mastra/core	Background execution. The workflow starts without blocking, and you consume chunks through PubSub.
createInngestAgent()	@mastra/inngest	Production deployments. Inngest adds step memoization, retries, and a monitoring dashboard.

All three return an object you register with Mastra the same way as a regular agent. createDurableAgent() and createEventedAgent() return class instances that extend Agent. createInngestAgent() returns a Proxy-backed object that forwards Agent methods to the underlying agent. When a signal wakes an idle thread, all three start that run with the durable stream(). sendSignal() and sendNotificationSignal() both work this way.

In-process with createDurableAgent()

Wrap your agent and call stream(). You get a DurableAgentStreamResult back in the same process. No external infrastructure is required, so this is the fastest way to get started:

src/mastra/agents/durable.ts

import { Agent } from '@mastra/core/agent'
import { createDurableAgent } from '@mastra/core/agent/durable'

const agent = new Agent({
  id: 'helper',
  instructions: 'You are a helpful assistant.',
  model: 'openai/gpt-5.6-sol',
})

export const durableHelper = createDurableAgent({ agent })

Fire-and-forget with createEventedAgent()

The workflow starts in the background without blocking the caller. You still receive chunks through PubSub, so stream() returns a result you can consume. The HTTP handler that triggered the run doesn't need to wait for the workflow to finish:

src/mastra/agents/evented.ts

import { Agent } from '@mastra/core/agent'
import { createEventedAgent } from '@mastra/core/agent/durable'

const agent = new Agent({
  id: 'writer',
  instructions: 'You write articles.',
  model: 'openai/gpt-5.6-sol',
})

export const eventedWriter = createEventedAgent({ agent })

Stored agents created through the API

Agents created with createStoredAgent() opt in with a durable field on the agent config. The server wraps the agent with createDurableAgent() when it hydrates it, so no code deployment is needed:

await mastraClient.createStoredAgent({
  id: 'helper',
  name: 'Helper',
  instructions: 'You are a helpful assistant.',
  model: { provider: 'openai', name: 'gpt-5' },
  durable: true,
})

durable also accepts { maxSteps, cleanupTimeoutMs }. Only serializable options are accepted here, so snapshot persistence for API-created agents follows the server's recovery.durableAgents setting. Cache and pubsub are inherited from the server's Mastra instance, so configure distributed backends there if you need durability across replicas. Automatic recovery is still configured in code through recovery.durableAgents.

Inngest-powered with createInngestAgent()

Run the workflow on the Inngest platform. Each tool call becomes a memoized step that Inngest can retry independently, and you get a dashboard for monitoring runs:

src/mastra/agents/inngest.ts

import { Agent } from '@mastra/core/agent'
import { createInngestAgent } from '@mastra/inngest'
import { Inngest } from 'inngest'

const inngest = new Inngest({ id: 'my-app' })

const agent = new Agent({
  id: 'analyst',
  instructions: 'You analyze data.',
  model: 'openai/gpt-5.6-sol',
})

export const inngestAnalyst = createInngestAgent({ agent, inngest })

Visit the createInngestAgent() reference for the full API, including Inngest-specific options like PubSub and cache configuration.

Resumable streams

Durable agents support resumable streams through PubSub and an event cache. When a client disconnects mid-stream, the cache continues storing events. The same client can reconnect by calling observe() with the runId:

src/server/reconnect.ts

const { output, cleanup } = await durableResearcher.observe(runId)

for await (const chunk of output.fullStream) {
  // Chunks from the run, including any missed while disconnected
}

cleanup()

createDurableAgent() and createEventedAgent() use an in-memory cache by default, which means resumable streams work within a single process. For production, provide a persistent cache backend (e.g., Redis) so cached events survive process restarts:

src/mastra/agents/durable-with-cache.ts

import { createDurableAgent } from '@mastra/core/agent/durable'
import { RedisServerCache } from '@mastra/redis'
import Redis from 'ioredis'

const cache = new RedisServerCache({ client: new Redis('redis://localhost:6379') })

export const durableAgent = createDurableAgent({
  agent,
  cache,
})

createInngestAgent() doesn't enable caching by default. Pass a cache option or register the agent with a Mastra instance that has a serverCache configured to enable resumable streams.

For cached topics, each published event is recorded in the cache before it's delivered live, so publish latency is bounded by the round-trip to the cache. If the cache write fails, the event is still delivered live but can't be replayed. @mastra/redis and @mastra/valkey record each event in a single round-trip using a Lua script. @mastra/redis falls back to separate commands when the client has no evalScript or when Redis Cluster rejects the multi-key script. Per-run workflow watch events (workflow.events.v2.*) are never cached. If the cache is remote (for example, in another region) and you don't need to resume a topic, use shouldCache to publish that topic straight through:

src/mastra/agents/durable-hot-topics.ts

export const durableAgent = createDurableAgent({
  agent,
  cache,
  // Skip the replay cache for the per-chunk stream topic; other topics stay resumable.
  shouldCache: topic => !topic.startsWith('agent.stream.'),
})

Streaming with background tasks

Durable agents support the same untilIdle option as regular agents. When untilIdle is set, stream() keeps the connection open across background-task continuations until the agent is idle:

src/mastra/run.ts

const { output, cleanup } = await durableAgent.stream('Research and summarize the topic', {
  untilIdle: true,
  memory: { thread: 'thread-1', resource: 'user-1' },
})

for await (const chunk of output.fullStream) {
  // Chunks from the initial turn AND any follow-up turns triggered by
  // background task completions
}

cleanup()

Pass { maxIdleMs } to customize the idle timeout (defaults to 5 minutes):

await durableAgent.stream('Research topic', {
  untilIdle: { maxIdleMs: 30_000 },
  memory: { thread: 'thread-1', resource: 'user-1' },
})

Visit Background tasks for the full background task guide, including configuration, subagents, and suspend/resume.

Cleanup

Every stream() and observe() call returns a cleanup function. Calling it unsubscribes from PubSub and removes the run from the internal registry. If you forget to call it, an automatic timer fires after the stream ends, but calling cleanup() yourself frees resources immediately.

Tool approval

Durable agents support tool approval (human-in-the-loop). When a tool call requires approval, the workflow suspends, emits an onSuspended callback, and waits for the caller to resume with resume():

src/mastra/run.ts

const { output, runId, cleanup } = await durableAgent.stream('Delete the old records', {
  requireToolApproval: true,
  onSuspended: ({ toolCallId, toolName, args }) => {
    // Notify the user and ask for approval
  },
})

Resume the suspended run after approval:

await durableAgent.resume(runId, { approved: true })

Crash recovery

Durable agents can checkpoint each run's state to storage while it executes. If the server process crashes mid-run, the run remains in running status in storage with no automatic retry, and on the next server start you can re-drive these orphaned runs so they pick up where they left off. For orderly shutdowns such as rolling deploys, the generated server can also drain in-flight turns before exiting. See graceful shutdown and rolling deploys.

Under the default persistence policy, these running checkpoints are only written when crash recovery is enabled. With recovery.durableAgents: 'off' (the default), durable agents persist snapshots only for pending, paused, and suspended runs (the artifacts human-in-the-loop resume depends on) and skip the per-step running writes entirely. A custom shouldPersistSnapshot predicate can keep running checkpoints even with recovery off. See snapshot persistence for the full policy and how to override it.

Durable agent runs are excluded from the generic boot-time restart of active workflow runs. The only automatic recovery path for durable agent runs is recovery.durableAgents: 'auto', which holds a recovery lease and registers thread runtimes before re-driving each run.

Snapshot persistence

The shouldPersistSnapshot option on createDurableAgent() (also accepted in the agent-level durable config) controls which workflow snapshots a durable agent writes. The default policy:

• Always persists pending, paused, and suspended snapshots. These are what resume() and tool approval read, so human-in-the-loop flows work without configuration.
• Persists running checkpoints only when recovery.durableAgents is 'auto'. These per-step writes exist solely to make in-flight runs recoverable after a crash, so they're skipped when nothing consumes them.

To keep crash-recovery checkpoints without enabling automatic recovery (for example, when you call recoverActiveRuns() yourself behind a leader election), pass a predicate that includes running:

src/mastra/agents/durable.ts

export const durableResearcher = createDurableAgent({
  agent,
  shouldPersistSnapshot: ({ workflowStatus }) =>
    ['pending', 'paused', 'suspended', 'running'].includes(workflowStatus),
})

Mastra logs a warning when a custom predicate excludes suspended or paused, which breaks human-in-the-loop resume, or excludes running while recovery.durableAgents is 'auto', which makes the agent invisible to automatic recovery.

Evented agents always persist the full snapshot set: the evented engine coordinates workers through storage, so the running row is part of its execution model. Inngest agents always persist only suspended snapshots, because Inngest's own replay provides durability. Both accept shouldPersistSnapshot for API symmetry but log a warning and ignore it.

Automatic recovery

Set recovery.durableAgents to 'auto' in the Mastra config. The deployer calls recoverAllDurableAgents() on boot, right after restarting active workflow runs:

src/mastra/index.ts

export const mastra = new Mastra({
  agents: { myAgent: durableAgent },
  storage: new PostgresStore({ connectionString: process.env.DATABASE_URL! }),
  recovery: { durableAgents: 'auto' },
})

On startup, this discovers every registered durable agent with runs stuck in running status and re-drives them from the last persisted snapshot.

warning

Recovery re-runs the agentic loop from the last snapshot, which re-issues LLM calls (real cost) and re-executes tool calls. Make sure your tools are idempotent before enabling automatic recovery.

Manual recovery

If you need finer control, such as gating recovery behind a leader election or running it on a schedule, call the methods directly:

// Recover all durable agents
const result = await mastra.recoverAllDurableAgents()
console.log(`Recovered ${result.recovered} runs (${result.succeeded} ok, ${result.failed} failed)`)

// Recover a specific agent
const agentResult = await durableAgent.recoverActiveRuns()

// Recover a single known run
await durableAgent.recoverActiveRuns({ runId: 'run-abc-123' })

Manual recovery reads the same running checkpoints as automatic recovery. If recovery.durableAgents is 'off' and you haven't set a custom shouldPersistSnapshot that includes running, in-flight runs are never checkpointed, so listActiveRuns() and recoverActiveRuns() find nothing after a crash. See snapshot persistence.

Multi-instance deployments

Mastra doesn't provide a distributed lease or lock yet. In multi-replica deployments, every replica that starts with recovery.durableAgents: 'auto' will race to recover the same runs. For now, either gate recovery behind your own leader election or run it from a single replica.

Related

• DurableAgent reference
• createInngestAgent() reference
• Background tasks
• Inngest deployment guide
• Agent overview
• Worker overview