Skip to main content

How to run long-running AI agents without starting over

Rafael Torres
Rafael TorresSeptember 11, 202616 min. read
How to run long-running AI agents without starting over

The long-running AI agent does not fail at the first step. It fails at step 40, when it has already opened the ticket in the support system, notified the customer by email, and launched provisioning in the ERP. The whole process has already happened. Then the model provider goes down, the run dies, and someone on the operations team types the most expensive sentence in enterprise automation: run it again from the beginning.

The operating contract that eliminates that sentence has five items: checkpoint per step, error classification between resumable and fatal, idempotency of external effects, human approval stops, and versioning of in-flight executions. It is written into the agent design, verified in the pilot, and delivers a measurable result: the failure that resumes instead of restarting.

The industry has just admitted that durability is an architecture decision. On August 27, 2026, Vercel published The best workflow engine is a programming language, by Pranay Prakash, the argument that durable execution belongs in the code itself. The decision got its manifesto. What is missing is the next layer: the contract that makes durable execution survive the third month, when the run is parked on a four-day approval and the code has already shipped three versions.

What is a long-running AI agent?

A long-running AI agent is one that executes a process spanning hours, days, or weeks: customer onboarding, procurement approval, recurring billing, payment disputes. It is not a long conversation. It is a stateful process that touches external systems and needs to survive partial failures without losing what it has already done.

A chatbot errs and repeats the answer. A long-running agent errs and the process is left half done: the customer notified, the ticket open, provisioning launched, the payment still pending. The question and answer architecture lives on requests that take seconds; the run of an AI agent in production outlives the connection and, when a deploy arrives before the end, it outlives the version of the code that started it.

The operating contract has five items, one per step of this guide:

  1. Checkpoint per step. The state each step commits before the next one: inputs, outputs, and decision. Resumption starts from the last checkpoint, never from zero.
  2. Error classification. Every failure gets a destination written before it happens: resumable comes back with backoff within a budget; fatal ends with consistent state.
  3. Idempotency of external effects. Payment, email, and ticket may already have gone out when the failure arrives; none of them repeats on the way back.
  4. Human approval stop. The run parks for minutes or weeks and stays alive, resuming from the same checkpoint when the decision arrives, without losing what was committed.
  5. Versioning of in-flight executions. The code changes; the run in flight finishes on the version that started it.

Without the five, every failure charges the whole process.

With them, a failure costs one detour.

The adoption map is in the B2B agents guide. The slice almost no pilot tests is this one: what happens when the run breaks in the middle.

What needs to be decided before step 1?

Before the first checkpoint, four decisions need to exist: the inventory of the process's external effects, the minimum state each step carries, who approves what, and where the state lives. Without that map, the checkpoint records what does not matter and ignores exactly what resumption would need.

First decision, the inventory of external effects: everything the process does outside the machine itself, API call, payment, email, ticket, provisioning. Each line becomes an idempotency candidate in Step 3 and shows where failure hurts.

Second decision, the minimum state per step: what the next step needs to know to start. Storing more creates a huge log and a versioning problem; storing less breaks resumption.

Third decision, who approves what: the permission matrix of the process, defined before the design, never during the incident. Fourth, where the state lives.

The decision of where durable execution lives, dedicated engine or execution written in the code itself, already has an owner on this blog: the post Durable execution for AI agents: the engine lives in the code covers the whole debate. This guide does not re-argue it. It assumes the decision is made and builds the layer it does not deliver on its own: the operation that turns promised durability into observed durability.

Step 1: define the checkpoint of each step

The checkpoint is the state each step commits before calling the next one: inputs received, outputs produced, decision made. Defined per step, it turns failure into resumption. The run comes back from the last checkpoint, not from zero, and what was already done stays done.

The commit rule is simple and non negotiable: the checkpoint writes before the external effect, and the result of the effect writes as soon as it returns. Between the two, the run is in flight, and it is that interval the next two steps protect.

The committed state carries, per step, the inputs received, the outputs produced, and the decision made, including the data the business team audits six months later. Minimal state is not poor state: it is what resumption consumes, not what the log collects.

Think of the checkpoint as the autosave of a text editor. Nobody writes 40 pages without autosave. The difference: here the autosave does not store a draft, it stores a commitment, the record of what the agent has already promised to the rest of the company.

That same per-step record is the raw material for measuring intermediate steps when the agent goes into production: the data that serves resumption serves evaluation.

inline-01.png

Step 2: classify every error as resumable or fatal

Every failure of a run needs a destination written before it happens: a resumable error comes back with backoff within a retry budget, a fatal error ends with consistent state and signaling. A transient that exhausts the budget is reclassified as fatal. No run loops without a ceiling: every run ends in resumption, completion, or signaled termination.

Taxonomy before code. A transient error is what repetition solves: network timeout, provider rate limit, momentary unavailability. A fatal error is what repetition makes worse: invalid input from the source system, violated business rule, inconsistent data that no attempt fixes.

The retry budget is the piece rushed designs skip. Exponential backoff without a limit is not an error policy, it is hope with a timer. The budget declares how many attempts and how much cost the transient gets; once the budget is spent, the error becomes fatal and the run ends with consistent state and signaling. Error does not leave the system through silence.

The contract becomes auditable when each class gets a row:

Error classExampleAgent actionEffect on the process
ResumableNetwork timeout on the provider callRetry with backoff, within the retry budgetNone: the last checkpoint stays valid
ResumableRate limit of the model providerRetry with backoff, same budgetThe step re-executes from the last checkpoint
Transient exhaustedTimeout that consumed the whole budgetReclassify as fatal and terminateConsistent state, signaled termination
FatalInvalid input from the source systemPause and signal the responsible teamNothing executes on bad data
FatalBusiness rule violated, credit limit reachedTerminate with the reason recordedTraceable process, no partial effect

The classification lives in the agent's code, one decision per exception type. A new error in production enters the table before it enters the handling. An undeclared class is a class decided by chance.

Step 3: make every external effect idempotent

An idempotent external effect produces the same result executed once or twice: the single payment, the single email, the single ticket. The idempotency key derives deterministically from the run plus step pair and is never regenerated per attempt, and the agent checks the previous effect in the external system before writing again.

The hole sits between two operations: the external call goes out, the network drops, the checkpoint never got written. On the far side, the payment happened. On this side, the state says nothing happened. Resumption re-executes the step, and the customer receives the second charge for the same purchase.

The idempotency key closes the hole, and here idempotency in AI agents stops being payments jargon. It derives from the run plus step pair, deterministically: same run, same step, same key, on every attempt. A key regenerated per attempt defeats the mechanism, because every attempt looks like a new effect to the external system. With a stable key, repetition becomes a question: does the effect with this key already exist? If it exists, the agent consumes the recorded result and moves on. If it does not, it executes once and records it.

Checking the external effect before writing again complements the key, it does not replace it. The key decides the identity of the effect; the check covers the external system without native deduplication. A duplicated and reversed charge is an operational loss. A duplicated and undetected charge is an accounting loss, the kind that shows up at month close with nobody's last name on it.

inline-02.png

Step 4: design the human approval stop

The human approval stop is a state of the run, not an exception: the long-running AI agent parks for minutes or weeks, the process stays alive, and it resumes from the same checkpoint when the decision arrives. The design defines who approves what, an expiration deadline, and an escalation route, with an audit trail of every decision.

The run parks. That is the design working, not a freeze: real B2B processes wait for credit approval, legal validation, customer confirmation. The difference between a designed stop and a lost run sits in three decisions: how data enters and leaves a running execution, how resumption happens, and what happens when the decision never arrives.

Data passes through the resumption mechanism the platform exposes, webhook or hook. The callback contract does not change: the decision arrives signed, with the identity of who approved, a timestamp, and the payload the next step consumes. That trail is the audit of the process, designed together with the permission, not afterwards. The layer on approvals and traceability of agent actions details who authorizes what and how every action gets recorded.

The deadline exists because approval without expiration is a process in limbo. The design declares how long the stop waits and where it escalates when the deadline passes: another approver, another channel, a reminder run. It is the fix for the classic failure of human approval in AI agents, the approval parked with no way back.

Step 5: version without breaking in-flight executions

Changing an agent's code while executions are in flight breaks resumption when the new code reinterprets the state written by the old one. The run in flight needs to finish on the version that started it. Every run stays pinned to the version that created it, and the new version applies to runs that start afterwards.

The mechanism behind it is deterministic replay: resumption reconstructs the path traveled from the committed state, and the reconstruction only makes sense with the code that wrote it. Version A writes a checkpoint with three fields. Version B expects five. The replay does not break loudly; it drifts, and decides with a field that did not exist. Corrupted state in silence is the worst state a system produces.

The industry converged on the same answer, with different names. The concept is generic. Pin each execution to the version that started it: the run carries the version marker, the old version stays executable while runs exist on it, and the migration of parked runs is an explicit decision, with a replay test between versions. The deploy only affects the runs that are yet to be born.

How do you test that the agent actually resumes?

The test that proves resumption is fault injection: kill the run in the middle of a step, confirm it comes back from the last checkpoint, check that no external effect duplicated, and verify the final output is the expected one. The test includes checking that the idempotency key is the same before and after the injected failure.

Reliability of AI agents is not declared, it is demonstrated. The suite below runs in the test environment before the pilot and runs again after every version change. The run dies on purpose:

  1. Kill the run in the middle of a step that has already called an external system, with the checkpoint not yet written.
  2. Confirm resumption from the last committed checkpoint, without re-executing previous steps.
  3. Verify zero duplicated effects: one payment, one email, one ticket, each effect existing exactly once.
  4. Check the stability of the idempotency key: the key generated before the injected failure is the same after it.
  5. Inject a fatal error and confirm termination with consistent state and correct signaling.
  6. Inject an approval stop, resume through the callback, and validate the expected final output.

An agent that passes the six tests actually resumes. An agent that fails item 3 has no operating contract; it has a long function with hope embedded. Failure recovery in AI agents is this suite running on every version, not a paragraph in the documentation.

Which failures still break long-running agents?

Four failure modes survive when the contract is badly applied: an effect executed twice after resumption, an approval parked with no way back, a code update that breaks the run in flight, and silent state drift when the external system changes between the failure and the resumption. Each mode has a known fix.

Duplicated effect after resumption. The classic cause is the key regenerated per attempt, the anti-pattern of Step 3, or a checkpoint written after the effect with no key at all. Fix: deterministic key from the run plus step pair, plus the check of the external effect before writing again.

Approval parked with no way back. The run stopped, the approver changed teams, and nobody knows the wait became abandonment. Expiration deadline and escalation route, defined in Step 4, turn indefinite waiting into a queue with an owner.

Code update that breaks the run in flight. The deploy goes up, the checkpoint from the previous version arrives at the new version, and the replay decides with a field that did not exist. Fix from Step 5: pin the run to the version that started it, with a replay test between versions.

Silent state drift. The external system changed between the failure and the resumption. The stock ran out. The credit policy changed. The customer canceled the order. The checkpoint describes a world that no longer exists, so the fix is to revalidate the step's preconditions at resumption and treat the divergence as a classifiable error, not as a loose exception.

What should you require from a B2B agents platform?

The operating contract becomes a buying checklist: ask how the platform stores state per step, how it classifies and reclassifies errors, how it sustains idempotent effects, how it keeps a run alive during an approval that takes days, and how it protects in-flight runs during an update. The answer that counts gets verified in the pilot, not on the brochure.

In practice, the B2B buyer has two implementation environments inside Nexforce Agents. Nexforce Work is the desktop workspace for the business team: multi-workspace orchestration, approvals and permissions layer, sandbox execution, scheduled runs, reusable workflow templates, and MCP connectors. Nexforce Code is the runtime for the engineering team: agents in the terminal and in the IDE, headless runs for automation and CI, agents and subagents per project, skills and MCP support, on macOS, Windows, and Linux.

Underneath both, the Nexforce Router operates the model layer: routing, failover, and cost governance of every call. The platform takes care of model access; the operating contract remains process design, because approval with a deadline, the idempotency key, and version pinning belong to the agent design and get verified with fault injection.

The rest is sales folklore.

FAQ: long-running AI agents

These are the questions that show up when the operating contract gets designed, answered directly. Each one maps to a step of this guide: definition and state, proof of resumption, approval stops, idempotency of external effects, and versioning of in-flight executions. Failures get destinations, effects do not repeat, and in-flight runs finish the version that started them.

What is a long-running AI agent?

It is the agent that executes a stateful process spanning hours, days, or weeks, touching external systems at every step: onboarding, approvals, billing, disputes. Unlike a conversation, the run needs to survive partial failures and approval stops without restarting the whole process.

How do you prove that the agent resumes instead of restarting after a failure?

With fault injection testing: kill the run in the middle of a step, confirm resumption from the last checkpoint, verify the idempotency key is the same before and after the failure, and check that no external effect duplicated. Proof is an observed result, not a promise.

Can AI agents wait for a human approval for days?

They can. The wait is a designed state of the run, not a freeze: the process stays alive, with a checkpoint, an audit trail, and resumption through the callback when the decision arrives. An expiration deadline and an escalation route keep the wait from becoming silent abandonment.

What is idempotency and why do AI agents need it?

Idempotency is the property of an effect that produces the same result executed once or twice. AI agents need it because resumption after a failure never knows whether the payment, the email, or the ticket already went out; the deterministic key from the run plus step pair prevents the duplicated effect.

How do you update an agent's code with executions in flight?

By pinning each execution to the version that started it: the run in flight finishes on the old version, with replay tested between versions, and the new version applies only to runs that start afterwards. Without this, the new code reinterprets state written by the old one and corrupts the decision.

References and further reading

The contract comes before the pilot

The position fits in one line: in long-running AI agents, the failure is not the problem, the restart is. The restart is a design decision, not bad luck: five contract items, a fault injection suite, and a pilot that tests the middle of the process.

The cost asymmetry decides the rest. Resuming costs writing one checkpoint and the discipline of testing with injected failure. Restarting costs the whole process, the three systems already touched, and the patience of the customer who received the second charge. When the pilot does not inject failure, it measures the happy path and hides the cost of the restart in the pilot's ROI, the bill that shows up after the scaling decision.

That is why the contract comes before the pilot: by the time the run falls, there is no time left to sew the parachute.

Nexforce

Deploy Work and Code Agentswith zero software licensing costs

Automate operational tasks and code writing autonomously with dedicated agents integrated into your systems

Free Trial

Related articles