What are durable AI agents? How state persistence and checkpoints survive a restart

From a simplistic perspective, an agent is a language model in a loop. The model's initial response is fed back to it as input, and it can react to a mistake it made, set new subtasks, and keep going without you giving additional instructions. Give it tools like web search, code execution, and file edits, and it can take a loose goal and grind on it for a while.

That "for a while" part is the basis of durable AI agents.

Let's break down what a durable or long-running agent means, what the challenges are in making such an agent, and how we implemented long-running AI agents in Mastra.

October 28, 2026 clarification: The example below continues an intentionally suspended workflow from persistent storage. Mastra also ships a dedicated durable-agent API, currently in beta, which wraps the agent loop in a workflow and supports in-process, evented, and Inngest execution. Distributed runs must be configured with shared storage and PubSub/cache backends across processes; the default event cache is in-memory.

What is a long-running or durable AI agent?

A long-running agent makes progress on a goal across many sessions and many machines, over hours or days, recovering from persisted checkpoints within its runtime's recovery contract.

There are a couple of ways you can think of long-running as a concept.

• An agent could execute for a long time. That could be multiple hours or even days.
• It could think for a long time before responding to queries, which we saw when the reasoning effort became a customizable option.
• Then we have agents with persistent memory, where the agent comes back to where we left off, keeps accumulating memories, and continues a task without asking for the same context over and over again.

It's not fair to call these agents chatbots anymore.

An AI travel agent, for instance, can plan the itinerary, wait for a manager to approve the expense, then finish the booking on a different server than the one it started on.

But even though it all sounds cool on paper, long-running agents have three problems:

• Context is finite, so the loop eventually runs out of room.
• State is not persistent, so a restart wipes everything the loop held in memory.
• The agent cannot reliably check its own work, so it needs a record outside itself to know what actually happened.

What keeps an agent running for days?

An agent can run for days only when the part holding its memory is separate from the part doing the work. The durable setups being built now split the job three ways:

• The model loop decides the next move. It keeps nothing of its own, so you can kill it and start a fresh one between steps.
• The sandbox is where tools run and files get written. It can crash or be swapped anytime, because it holds no truth either.
• The durable record is the log of what happened. The one source of truth a fresh model loop reads to pick up where the last one stopped.

With persisted state and a recovery mechanism, a replacement process can continue from the recorded progress. That makes a long-running agent less one program running for a week and more a relay of processes passing a record between them. Sandbox files also need durable storage if later work depends on them.

This is why the durable record is the most important part here. The rest of this piece is about that record: why context windows cannot be durable, what goes into the record, and how a fresh process reads it back.

Why is a context window not a persistence layer?

People reach for the context window as if it were the agent's memory. But it fails as a persistence layer because:

• The window is finite and lossy. When it fills, the agent compacts history into a summary, and compaction throws away the detail the agent needs later.
• A fresh session starts blank. The loop holds its progress in memory, so when the worker restarts, the in-memory loop comes back knowing nothing about the run that was halfway done.

What are the three granularities of persistence?

The right way to think about persistence is to ask what the smallest unit of work is that you refuse to lose. I've found there to be three answers, depending on what the agent is doing.

1. Turn-level persistence keeps a conversation alive between messages

The unit is one turn, a single back-and-forth with the user, and progress is driven by the person sending the next message.

Between turns, the agent saves the conversation and what it has learned, so it still knows what you said three turns ago and what it picked up about you last week. Losing a turn is cheap, since you just say it again, which is why a plain chat assistant only needs this layer.

2. Step-level persistence records each completed step in a workflow

The unit here is one step inside a single run (or turn), a node in a workflow. Persisting completed step outputs lets a recovery mechanism reuse that progress instead of starting from the top.

A single turn can kick off one of these runs, so step-level sits underneath turn-level rather than beside it. What makes it a different problem is the cost of losing a step, because a step that charged a card or called a paid API cannot simply be rerun. An API pipeline that charges, then reserves inventory, then emails a receipt needs idempotency keys or equivalent safeguards as well as checkpoints: a crash between an external action and its recorded result can otherwise repeat that action.

3. Mid-step persistence holds a pause inside a single step

The unit here is a saved suspension inside a step, waiting for input that might arrive days later. No model loop runs during that wait, so a paused approval costs nothing but idle storage.

An approval gate needs this. In Mastra, resuming re-executes the suspended step with resume data; it does not restore the JavaScript call stack at the suspension point.

How does a long-running agent survive a crash?

Addy Osmani put it well: the agent is amnesiac, but the filesystem is not.

To put that into practice, progress has to be written down in units. A useful unit is "the step." Persisted step results provide recovery points; the runtime still needs a way to detect failure and restart execution.

With storage configured, Mastra writes a snapshot when the workflow suspends (more on that in the next section), keyed by a run ID. It holds recorded step state, suspension payloads, and the run's input as serializable data.

For this saved-suspension example, a replacement process loads the run by ID using the same workflow definition and storage.

// brand-new process: nothing from the original run is in memory
const wf = mastra.getWorkflow('orderWorkflow');
const state = await wf.getWorkflowRunById(runId);
const reader = createWorkflowStateReader(state);
 
const suspended = reader.getSuspendedStep();   // { stepId: 'human-approval', ... }
const run = await wf.createRun({ runId });      // bind a run to the SAME id
await run.resume({ step: suspended.stepId, resumeData: { approved: true, approver: 'manager-jane' } });

I ran the snippet above from a fresh Mastra instance that shared only the database file, with nothing from the first process in memory. The run came back suspended at the approval step, still holding the reservation computed before the pause, and the resume drove it to completion.

The agent didn't have to "remember" anything, because it could rebuild the full execution state up to that point from just the ID.

How does checkpointing work in Mastra, and what gets serialized?

Now let's go a little deeper into the checkpointing aspect of Mastra. With persistent storage configured, suspension writes a serializable snapshot of the workflow state. After that write succeeds, a later process with the same workflow definition and access to the store can load the suspended run. This preserves the recorded state, not arbitrary in-memory objects or external side effects.

A snapshot captures the whole run

I ran an order workflow that reserves inventory, waits for a manager to approve, then ships, and suspended it at the approval. Reading the snapshot straight back out of storage by run ID, here is what it holds:

• Per-step status, one of success, suspended, failed, waiting, running, or paused.
• Each completed step's recorded output, so the reservation ID computed before the pause is sitting right there.
• The execution path through the graph, as a serialized step graph.
• The suspend and resume payloads for any paused step.
• The remaining retry attempts for each step.
• The run's original input and the shared workflow state.

The whole snapshot for that suspended run serialized to 1,059 bytes of JSON, with the per-step map reading reserve-inventory: success and human-approval: suspended, and a suspended-paths entry pointing at the approval step.

Every step entry carries its own four timestamps

Mastra has a per-step record, which is interesting because it tells the engine what state a step is in without rerunning the step.

Let me explain. When the run suspends a step, the entry status reads suspended, and it carries a startedAt and a suspendedAt timestamp with a suspend payload describing why the step stopped.

After I resumed the step, that same entry reads:

{
  "status": "success",
  "startedAt": 1782382177053,
  "suspendedAt": 1782382177053,
  "resumedAt": 1782382177060,
  "endedAt": 1782382177060,
  "resumePayload": { "approved": true, "approver": "manager-jane" }
}

The four timestamps here, startedAt, suspendedAt, resumedAt, and endedAt, are a full audit trail for that single step, and the resume payload is the human decision, captured as plain data the moment it arrived.

The decision could arrive days later; with the snapshot retained, resuming re-executes the suspended step with that decision.

The whole snapshot has to convert to JSON

Everything in the snapshot is serialized as JSON. So if a step needs a 50 MB document, the agent should only keep the document's ID in its state and fetch the actual document when it's required.

Keep step state small and made of plain values, and the snapshot stays small, fast to write, and safe to reload anywhere.

Why can't a long-running agent trust its own work?

It's like letting a student grade their own papers. If you let an agent verify its own work, the agent is unlikely to find any mistakes, because it was the one that went through the decision-making process.

The way out is to separate the doing from the judging. A worker produces the result, and a judge, a different model or at least a fresh prompt holding the explicit done conditions, checks whether the work was actually done as per the requirements.

Cursor's coding agents work similarly. There are planners that emit tasks, workers that execute, and judges that verify.

This setup puts all the weight on the done conditions that you write before an agent starts working.

Checkpointing and verification reinforce each other here. The judge reads the worker's recorded output from the durable record and writes its verdict back into the same record, so the decision to advance or retry survives a restart along with everything else.

The two states of a long-running AI agent

A long-running agent has to remember two different things.

Execution state answers "where in the work are we." It is the step status, the recorded output, and the pause payload in the checkpoint, scoped to one run and run ID.

Memory answers "what does the agent know": conversation history and retained facts. In Mastra, a thread identifies a conversation and a resource identifies a user or other entity. Cross-thread sharing depends on the configured memory mechanism and scope, not just using the same agent. Your application must authorize access and define retention and deletion policies.

They are not interchangeable, and a long-running agent needs both at once: the execution state to resume the task and the memory to stay the same agent across tasks.

Wrapping up

If we had to put it all together, I'd say a long-running agent is simply a durable record that short-lived processes take turns with.

The entire concept revolves around keeping the data out of the agentic loop. Because of this architecture:

• A persisted checkpoint gives a replacement process a recovery point. Work after that checkpoint and external side effects still need explicit recovery handling.
• A premature "done" gets caught, because a judge writes its verdict to the record before the run moves on.
• The agent stays itself across sessions, because what it knows lives in the record, not the process.

With Mastra you get execution, checkpointing, resuming runs, verification, and memory. You build the agent you need, and the architecture handles the rest behind the scenes.