Why replay and resume for AI agents must not be confused

Replay and resume use similar mechanics: both take a saved history and run code after a certain point. But they serve different purposes, and the cost of getting them wrong is different too. Replay is for going through the history again to understand why an agent made a decision. Resume is for a specific work operation that stopped and must reach completion without resending commands that have already run.

It is easy to combine both actions into one "Run again" button. Then the agent creates a CRM ticket twice, books the same product again, or sends the customer two emails with different wording. This is usually not a model error. It is a contract error between the orchestrator, the state store, and external tools.

Replay checks past history, while resume continues an obligation

Replay creates a new execution after a checkpoint. It is used to debug, test a new branch, analyze a model decision, or reproduce a defect from saved inputs. This run should be separate from the original process, with its own ID, launch reason, and tool-use mode.

Resume continues the same logical process. It already has obligations to the user and to external systems. If the agent stopped while waiting for approval of a transfer, resume should wait for the decision and run the remaining nodes. It should not classify the request again, create a new application, or reserve inventory a second time if those actions have already been confirmed.

It helps to make this explicit in the data model. One user conversation is not one run, one run is not one command, and a command is not the same as an external effect.

conversation_id = "conv_8841"
run_id          = "run_01J..."
command_id      = "cmd_send_offer_03"
effect_id       = "eff_run_01J_cmd_send_offer_03"

conversation_id connects messages. run_id describes one graph execution. command_id identifies the agent's intent within that execution. effect_id connects attempts to create one external result. If an engineer uses one thread_id for all four roles, the system can no longer answer a basic question: "Is this a continuation of the previous operation or a new attempt to do the same thing?"

LangGraph documentation draws the boundary clearly: replay runs nodes after the selected checkpoint and does not run the nodes before it because their results have already been saved. Later LLM calls, API calls, and interrupts can run again, however. This is not log viewing. It is a live code execution.

A state snapshot does not prove that an external effect did not happen

A checkpoint records the orchestrator's state. It does not turn the network into a transaction, and it cannot automatically know whether a third-party service accepted a request when the process failed.

Imagine a send_invoice node. The agent sends an HTTP request to a billing system. The billing service creates an invoice and returns 201 Created, but the process fails before the response is written to the checkpoint. After recovery, the orchestrator sees the old state, with an empty invoice_id. The external system sees the created invoice. If resume simply sends the request again, the customer receives two invoices.

This uncertainty window exists even in a carefully designed architecture:

1. the agent records its intention to perform an action;
2. the agent calls the external service;
3. the external service performs the action;
4. the response is lost, times out, or the process ends before the result is recorded;
5. the orchestrator tries to recover.

Logs help investigate the case, but they do not solve it by themselves. A log may show an outgoing request without proving that it was accepted. The external service's journal may show the effect, but the orchestrator cannot match it to an attempt without a shared identifier. You need a protocol that survives the gap between sending a request and saving its response.

For critical actions, store a separate effect-intent record before the network call. After the call, update that same record with the result. On the next attempt, the code should read its status first instead of assuming, "The checkpoint is empty, so nothing happened."

{
  "effect_id": "eff_run_01J_cmd_send_offer_03",
  "type": "crm.create_offer",
  "status": "pending",
  "request_hash": "sha256:...",
  "provider_reference": null,
  "created_at": "2026-07-23T09:14:06Z"
}

The statuses pending, accepted, completed, and failed are more useful than one boolean done field. pending means the process must check what happened before sending again. accepted means the recipient accepted the command, but the final result is not known yet. This distinction matters especially for payments, message delivery, and asynchronous tasks.

An idempotency key should describe the action, not the attempt

Idempotency means that delivering the same command again does not create a second effect. It does not mean you can repeat any function without thinking.

The most common mistake is generating a UUID inside the tool function. On the first attempt, the agent sends one key. On resume, it generates another, and the external service correctly treats the request as a new operation. This supposedly idempotent code protects only against an accidental duplicate within one HTTP client, not against process recovery.

The key must be created before the first attempt and live in the process state.

from hashlib import sha256

def effect_key(run_id: str, command_id: str) -> str:
    raw = f"{run_id}:crm.create_offer:{command_id}".encode()
    return sha256(raw).hexdigest()

async def create_offer(state, crm):
    key = state["effect_id"]
    existing = await crm.find_effect(key)
    if existing and existing["status"] == "completed":
        return {"offer_id": existing["offer_id"], "effect_status": "completed"}

    response = await crm.create_offer(
        customer_id=state["customer_id"],
        amount=state["amount"],
        idempotency_key=key,
    )
    return {"offer_id": response["id"], "effect_status": "completed"}

This example has two layers of protection. The internal check lets the agent find an effect that has already been recorded. The external idempotency_key asks the recipient to merge duplicate deliveries into one operation. Ideally, the recipient stores the key with a hash of the payload and rejects a repeat that uses the same key with different parameters. Otherwise, a routing error could turn the key "create offer" into permission to change the offer amount.

Not every tool provides an idempotency key. In that case, build deduplication yourself: create an effects table with a unique index on (effect_type, effect_id), record the intent transactionally, and check for a completed or pending record before the call. This does not remove the need to check with the external recipient if it could have executed the request before your failure. But it gives the system a place where the truth about the intent lives.

Idempotency is different from compensation. Canceling a reservation does not make the original reservation idempotent. It is a separate operation with its own risks, conditions, and log. Compensation is needed when the business allows a rollback. Protection against duplicate delivery is always needed when one logical request can arrive twice.

A pause for approval can run the node's code again

Human-in-the-loop often looks safe: the agent prepares an action, asks for approval, receives a response, and continues. Many runtimes, however, resume the entire node or function from a saved boundary rather than resuming a particular line of code.

The LangGraph documentation says that when interrupt() is resumed, the node starts from the beginning. Code before interrupt() therefore runs again too. The documentation recommends putting side effects after the interrupt, making preceding actions idempotent, or moving them into separate nodes.

A risky node can look harmless:

def approve_and_send(state):
    audit.create({"event": "offer_prepared", "run_id": state["run_id"]})
    decision = interrupt({"offer": state["offer"]})
    if decision["approved"]:
        mail.send(state["recipient"], state["offer"])

After resume, audit.create() runs again. If the audit uses append-only events without a unique key, you get two preparation events. That may be tolerable. It is much worse when crm.create_lead() or payment.authorize() appears before the pause.

A safer design separates the phases:

def request_approval(state):
    return interrupt({"effect_id": state["effect_id"], "offer": state["offer"]})

def send_approved_offer(state):
    if not state["approved"]:
        return {"status": "rejected"}
    return send_with_idempotency_key(state)

The first function shows a person the exact intent. The second calls the external service only after the process has recorded the decision. If your runtime can save task results, put the network call in such a task and still keep idempotency on the recipient's side. Caching the result reduces repeats in the normal path. It does not replace protection when a task starts and fails before being marked complete.

Do not wrap the pause mechanism in a broad try/except that swallows the internal stop signal. In LangGraph, an interrupt is implemented through a special exception and must reach the runtime. A caught pause often looks like "the agent declined the action" and then breaks the state for the next resume.

Replaying an LLM call is not the same as reproducing its response

Teams often call replay a "deterministic debugging run." For an agent, that is true only if you define exactly what is being reproduced.

If you save the request and return the recorded model response, you reproduce the decision-making history. This is useful when you need to understand why the agent called a tool or chose a route. If replay sends the same prompt to the model again, you are repeating an experiment. The response may differ, followed by changes in tool calls, action order, and the final result.

Even fixed parameters do not eliminate every difference. External search results change. Vector search changes after reindexing. A "get balance" tool returns current data. The provider may have deployed a new model version. Time, random numbers, and a network response inside control code can change the branch.

For debugging, mark every input as one of three types:

• a recorded fact that replay must return without a new call;
• a repeatable computation that can run again on the same data;
• a live request that deliberately accesses the current external world.

Replay needs an explicit mode. In evidence mode, the agent uses saved LLM responses, tool outputs, and documents. In simulation mode, test tools and isolated data copies are allowed. In live mode, real calls are allowed, but the run must be new and irreversible commands must be blocked unless explicitly permitted.

The word "replay" without such a mode is dangerous. An engineer may think they are opening a recording of a match, while they are actually sending the player back onto the field.

Branching history requires a new run_id and clear lineage

When a team changes a prompt, edits a value in state, or selects an old checkpoint, it creates an alternative history. It is not continuing the original one.

Store at least four fields for the branch:

{
  "run_id": "run_01K_new",
  "parent_run_id": "run_01J_original",
  "parent_checkpoint_id": "cp_0042",
  "launch_reason": "debug_after_tool_timeout"
}

This lineage is not about displaying a pretty graph. It prevents a dangerous mix-up. If an operator edits the amount, removes a document from context, or replaces a tool response, the result can no longer be treated as the result of the original request. A new branch can be an excellent way to test a fix. It must not send its action to the queue awaited by the original workflow.

In LangGraph, updating state creates a new checkpoint rather than changing the old one. This is the right model: the past remains available for investigation, while the new path gets its own starting point.

For the operator interface, separate commands by meaning:

• "Continue" is available only for a pause, failure, or pending action in the original run;
• "Replay from checkpoint" always creates a new run;
• "Create branch with changes" shows state differences and requires a reason;
• "Repeat external effect" is not hidden inside replay. It is a separate privileged command.

The last command should show the recipient, payload, existing effect_id, status of the latest attempt, and deduplication method. A "Run again" button provides none of this information. It therefore must not be used for an operation that changes the outside world.

An effect log matters more than a polished tool-call trace

A trace shows that the model asked to call a tool. An effect log shows what the system did in the outside world. In production, the second record is usually more important.

Each record should include an effect ID, command type, normalized request hash, attempt times, recipient ID, final status, and a link to the run or checkpoint. Store the full payload only where your access and data-retention rules allow it. For personal data, an encrypted protected log and a masked operator view are often enough.

Do not treat "the tool returned 200" as the status of the business effect. The API may have accepted a task without completing it. For example, document delivery may return accepted, while the message is sent later or rejected by the recipient's policy. The agent needs a clear contract for every tool: whether the effect is synchronous, how to learn its final status, whether it can be queried by effect_id, and how long the service retains the deduplication key.

Test this with failure scenarios, not by reading code. Choose a tool that creates a real test object. Stop the process deliberately after the request is sent but before the checkpoint. Resume should find the existing object by effect_id or retry with the same key while preserving one object. Then replay from the checkpoint in a test environment and confirm that it creates a new run instead of continuing the original.

If you cannot set up this test, your team has not yet defined recovery semantics. An orchestrator will not fix that.

Code versions can quietly break resume more easily than a network failure

An old checkpoint contains more than data. It assumes a particular node order, the meaning of state fields, and the order of approval requests. New code can violate those assumptions.

It is especially risky to insert a new interrupt before an existing one or reorder calls that the runtime matches to saved results. LangGraph documentation specifically warns that changing the order of tasks and interrupts before the resume point can associate a saved value with the wrong call. It also recommends letting unfinished processes complete, moving new logic into a new task, or starting a new entrypoint version.

A practical rule is simple: every workflow should have a workflow_version stored in the run. The runtime checks compatibility before resume. If the new version cannot read the old state, it does not guess. It offers state migration, execution with the old version, or manual review.

{
  "run_id": "run_01J...",
  "workflow_name": "sales_offer_agent",
  "workflow_version": 7,
  "state_schema_version": 4,
  "status": "waiting_approval"
}

Do not update a field so that its old meaning silently becomes a new one. The field approved must not suddenly mean "approved by a manager and passed the limit check." Add a new field and a migration. The state of a long-running agent is like a public contract, even if only your code reads it.

Retry policy should depend on the tool class

One global retry setting for every tool call is almost always wrong. Reads, writes, and irreversible actions carry different risks.

A catalog read or document search can be repeated, but the result may change. If the answer affects a decision, store the time and source version in the log. A write with a natural key can be repeated with proper deduplication. An irreversible action such as publishing a document, sending an external notification, or moving money needs a separate policy: check the existing effect, possibly request confirmation, and only then deliver the command.

A useful classification looks like this:

An LLM call is not automatically a pure computation. It does not directly change your database, but it can change the next decision. If the response is used only for a draft, repeating it is usually acceptable. If it determines a command to an external service, store the original response, tool-call arguments, and selected route as evidence of why the effect occurred.

AI Router can serve as one OpenAI-compatible gateway for model calls, but checkpoints, effect logs, and idempotency remain the application's responsibility. A proxy routes a request to a model. It does not determine what happens to your email, payment, or CRM record.

Remove the dangerous button first, then make the graph more sophisticated

If your interface has one "Retry" command, replace it before the next incident. For an unfinished operation, call the action "Continue." For an investigation, call it "Create replay." For redelivering an external command, create a separate operation with a visible effect_id and a status check against the recipient.

Then choose one tool with a real external effect and run the failure test described above. It will quickly show where the action identifier lives, what is saved before the call, what happens on a timeout, and whether an operator can distinguish a new branch from a continuation of an obligation.

An agent may reason unpredictably. The execution of its commands should not.