Cancelling Agent Generation and Saving the Partial Result

Cancelling an agent’s generation does not mean «throw away everything that happened». It means the system must stop further work and record the boundary honestly: what the user has already seen, what the server has confirmed, what external systems have accepted, and what can no longer be considered part of the result.

If the request after clicking «Cancel» continues from a point the user did not see, the problem is not the button. The problem is that the token stream, workflow state, and interface exist as three independent versions of the same story. In production, they must be brought under one contract.

Partial text and process state are not the same thing

A saved partial result must be a reproducible conversation state, not a random piece of a network stream. The browser may receive 800 characters, the server may already have generated 300 more, and the orchestrator may at the same time start a search, a CRM request, or payment preparation. If you write only what reached the screen into the history, the conversation will no longer match the actual execution state.

I usually separate data from a cancelled run into four layers:

• Visible result: messages and structured blocks published to the user, such as a table or a finished section of an email.
• Confirmed workflow state: the input-data version, completed nodes, results of finished tasks, selected branch, and checkpoint.
• Unfinished work: current generation, a pending tool call, a queued request, and background tasks.
• Execution traces: run ID, cancellation reason, time the command was accepted, and idempotency keys for external operations.

These layers must not be merged into one messages field. The text «I’m sending an email to the client» does not prove that the email was sent. The send_email API result does not mean the user saw the generated text. And a cancelled flag does not say whether the request was stopped before the external provider accepted it.

A common mistake is to save every incoming token directly into the main history. At first glance, this seems to solve everything: after cancellation, the full draft is available. In reality, you are saving an unstable artifact. The model may not have closed the JSON, finished a function call, or completed an intermediate formulation that it would have corrected in later tokens. When resumed, that text becomes false context, even though the user sees it as something the agent has already said.

Only a published block belongs in the main history. The stream can be stored separately as a short-lived technical buffer if it is needed for connection recovery or error investigation. But an agent resume must read the checkpoint and published messages, not the raw SSE or WebSocket buffer.

Cancellation needs its own protocol, not just a signal to the worker

A cancellation command must pass through the same layers as a run: interface, API, orchestrator, worker, tools, and state storage. An abort signal in the browser is useful, but it describes only the client’s wish to stop listening to the response. It does not confirm that the server has stopped the work.

A minimal status model looks like this:

{
  "run_id": "run_8f3c",
  "state_version": 17,
  "status": "running",
  "cancel_requested_at": null,
  "cancel_reason": null,
  "published_message_id": "msg_204",
  "active_operation": {
    "kind": "model_generation",
    "operation_id": "gen_771"
  }
}

After the user’s request, the server should not immediately return a final cancelled status if it does not yet know what happened. It should first atomically record the intention to stop the run:

{
  "run_id": "run_8f3c",
  "expected_state_version": 17,
  "action": "cancel",
  "reason": "user_requested"
}

A normal response to this command might be:

{
  "run_id": "run_8f3c",
  "status": "cancel_requested",
  "accepted_state_version": 18,
  "resume_checkpoint_id": "cp_18",
  "published_message_id": "msg_204"
}

The worker then checks the cancellation flag at safe points. For generation, this is the boundary between stream chunks. For a queue, it is the moment before taking the next task. For a tool, it is the moment before an irreversible action. Once the worker stops or finishes the operation it has already begun, it moves the run to a terminal status and records what remains available for resuming.

The Model Context Protocol task specification gives a useful rule: after an accepted cancellation, a task moves to cancelled and remains in that status even if execution actually finishes later. This is not a quibble about status names. It protects the interface from the worst scenario: the user cancels a task, sees «cancelled», and a second later the agent publishes a new response as if nothing happened.

For an agent application, I also add a cancelling status when an operation may take noticeable time to finish. It tells the truth: the server accepted the command, but cannot yet promise that the model provider or tool has stopped. Do not use cancelled as a decorative animation.

A checkpoint records a decision, not every character

A checkpoint should appear after a meaningful state transition. This may be a completed search, a confirmed plan choice, a published response, a received tool result, or a pause before an operation with consequences. It should not and need not be created after every token.

A checkpoint has three jobs. It provides a point from which work can resume. It separates completed results from work in progress. It lets the server tell the interface which version the user is allowed to edit or develop further.

A good checkpoint stores not the model’s private «thinking», but enough data for the continuation to make the same decisions that have already been confirmed:

{
  "checkpoint_id": "cp_18",
  "run_id": "run_8f3c",
  "parent_checkpoint_id": "cp_16",
  "state_version": 18,
  "conversation": [
    {"id": "msg_201", "role": "user", "content": "Сравни варианты поставки"},
    {"id": "msg_204", "role": "assistant", "content": "Я нашёл два применимых варианта...", "published": true}
  ],
  "completed_steps": ["extract_requirements", "search_catalog"],
  "tool_results": [
    {"call_id": "tool_91", "name": "catalog_search", "result_ref": "obj_667"}
  ],
  "pending": {
    "step": "draft_recommendation",
    "input_hash": "sha256:..."
  },
  "cancelled_generation": {
    "discarded_stream_chars": 412,
    "resume_policy": "regenerate_pending_step"
  }
}

discarded_stream_chars is not a product or analytics field. It helps an engineer understand why the user saw a shorter response than the server received from the model. The continuation itself must not take those 412 characters and append them to new text. It should rerun the unfinished step with the confirmed context.

LangGraph documentation draws the same boundary: the checkpointer saves stream state, while thread_id identifies which state to load on resume. It also contains an important detail that is often missed when designing cancellation: on resume, a node runs from the beginning, not from the exact line where the pause occurred.

The consequence is simple. If your «node» generates text, makes three tool calls, and sends an email, it is not a node that can be safely cancelled and resumed. It is several different state transitions mistakenly packed into one function.

Interface publication needs a version boundary

The interface must know not only the text, but also the version in which that text became visible. Otherwise, you cannot distinguish «continue this response» from «repeat the request after the state has already changed in another tab».

For every published block, keep at least four fields: message_id, state_version, publication_status, and run_id. When the user clicks «Continue», the client should send more than an abstract «finish this». It should reference a specific point:

{
  "thread_id": "thr_42",
  "resume_from_checkpoint_id": "cp_18",
  "expected_state_version": 18,
  "user_message": "Продолжи сравнение и добавь риски"
}

The server must compare expected_state_version with the current branch. If another tab has already continued the conversation and created version 19, the server must not silently mix the new request into the changed context. It returns a state conflict:

{
  "error": "state_version_conflict",
  "current_checkpoint_id": "cp_19",
  "current_state_version": 19,
  "allowed_actions": ["reload", "fork_from_cp_18"]
}

This response may seem strict until you see the alternative. The user cancels a report draft in browser A. In browser B, the agent finishes its search and gets new data. Browser A then sends «continue, but make it shorter». If the server accepts this without checking, the model continues a different conversation even though screen A shows the old version. The response looks coherent, but relies on facts the user has not seen or chosen.

Versioning does not require a complex branching system in the first release. A monotonic state number, a parent checkpoint, and an explicit conflict decision are enough. A branch is needed when the product truly allows both work paths to be saved, not because developers did not want to return 409.

Show users only actions the server can carry out honestly:

• «Continue from the saved point» when a usable checkpoint exists.
• «Change the request and continue» when the new message will become a child state of that checkpoint.
• «Start a new branch» when the current history has already moved forward.
• «Delete the draft» when the published partial text is not needed in the conversation.

The «Retry» button is too vague here. It may mean repeating a model request, repeating a tool call, continuing generation, or creating a new response. Do not make the user guess which of these four options the engineers implemented.

Tools need a boundary before irreversible action

Text generation can be stopped and restarted. External actions often cannot. If an agent creates a ticket, sends an email, changes a CRM record, books a slot, or publishes a document, cancellation must account for the operation’s phase.

A practical pattern has two parts: preparation and commit. During preparation, the agent gathers arguments, shows the user the action, and creates an intent record with an idempotency key. During commit, a separate worker performs the irreversible call. A checkpoint sits between them.

A bad order looks like this:

сгенерировать письмо -> отправить письмо -> показать пользователю текст -> ждать подтверждения

After cancellation, you can no longer say what was saved. The email may have been sent, but the user did not see its final version. On a repeat run, the agent may send a duplicate because the state contains no confirmed operation.

The workable order is different:

сгенерировать черновик -> опубликовать черновик -> сохранить checkpoint -> получить подтверждение -> отправить с idempotency_key -> записать результат

Even here, cancellation is not the same as retraction. If the provider accepted the send request before seeing the stop signal, your run status may become cancelled while the email still reaches the recipient. This is not contradictory if the interface says: «Cancellation accepted. Sending has already been handed to the external system; check the operation log.» A false guarantee is worse than an unpleasant but accurate message.

LangGraph documentation directly recommends placing side effects after an interrupt point or making them idempotent because a resumed node may run again. The example of creating the same database record twice is familiar: the code looks fine on the first pass and starts creating duplicate entities only after a pause, retry, or cancellation.

An idempotency key should describe the business action, not the execution attempt. For sending a quote, it might be quote:483:revision:7:send, rather than a random UUID for every call. A retry after a network timeout will then return the same result or show that the operation has already succeeded.

Not all cancelled data belongs in the history

Saving a partial result does not mean making it a permanent part of the conversation. Sometimes users cancel because the agent went in the wrong direction. Sometimes the stream contains incorrect details, invalid JSON, or a fragment that must not be shown to the next operator. Sometimes data-retention requirements prohibit keeping the raw output.

Define a policy in advance for four artifact types.

The most dangerous category is partially formed structured output. Suppose the agent was building JSON with order parameters and stopped after the quantity field. You cannot save it as a finished order card just because a parser extracted several fields. Either publish a clearly marked draft that cannot be used for execution, or discard it and continue from the last valid object.

The same applies to extracted data. If the agent showed three search records and the fourth arrived after cancellation, do not add the fourth invisibly on the next turn. Show that a set of three records was saved, and on resume either repeat the search or explicitly update the set. The user must understand what material the agent is using for its next conclusion.

For systems with local-storage requirements, one more level matters. State, the cancellation log, and the technical stream buffer have different lifetimes and sensitivities. Do not keep them in one store merely for schema convenience. In AI Router, commands can use one OpenAI-compatible endpoint, while data residency, PII masking, and audit-log requirements should be handled separately for each artifact instead of treating a cancelled stream as a harmless draft.

Connect the stream, checkpoint, and screen with events

A server stream is useful for responsiveness, but it must not be the only source of truth. Introduce events that let the client rebuild the screen from confirmed facts even if the connection drops and the user reconnects.

For example, the sequence might be:

{"type":"run.started","run_id":"run_8f3c","state_version":17}
{"type":"message.delta","message_id":"msg_204","seq":51,"text":"Я нашёл"}
{"type":"message.delta","message_id":"msg_204","seq":52,"text":" два варианта"}
{"type":"message.published","message_id":"msg_204","state_version":18,"checkpoint_id":"cp_18"}
{"type":"run.cancel_requested","run_id":"run_8f3c","state_version":19}
{"type":"run.cancelled","run_id":"run_8f3c","resume_checkpoint_id":"cp_18"}

A message.delta may be lost, arrive twice, or arrive after the user has already clicked cancel. The client may therefore show it as temporary text, but must not write it to permanent history without message.published. The publication event connects the screen to the checkpoint. The cancellation event says which point is available for the next intent.

If the product needs a «save what is visible right now» mode, make it a separate action. The client sends a request to commit the visible draft with the number of the last sequence, and the server validates it as an independent user artifact. Do not pretend that this fragment is a finished model response. The label «Saved fragment» removes ambiguity for both the user and the next run.

In large systems, it is useful to separate run_id and thread_id. The first describes one execution attempt. The second describes a conversation or work object. After a cancellation in one thread_id, a new run_id may continue from checkpoint 18. Trying to store both concepts in one identifier usually breaks auditing, retries, and support.

Resuming should repeat the intent, not glue on a sentence

When users write «continue», they rarely ask the model to mechanically finish a broken sentence. Usually, they want to return to the task using the work already completed. These are different things.

If cancellation happened during free-form text, resuming can rerun the same unfinished step. Give the model the confirmed context, state that the previous output was stopped by the user, and ask it to produce the next published block without relying on discarded tokens. The text will not be identical, and that is fine. Text determinism is not a guarantee worth pretending to provide.

If cancellation happened after a completed tool, resuming should use its confirmed result. Repeating a search or charging money again merely because the user stopped the explanation is an architectural defect. The state must record which step is complete and which is not.

If cancellation arrives during a tool call, the decision depends on its contract:

• The tool supports cancellation and confirms it. Mark the result as absent and repeat the work on resume.
• The tool does not support cancellation but is safe to repeat. Wait for its result or request it using the idempotency key.
• The tool has consequences. Confirm its status before any new run and show the user what happened.

Do not promise «resume exactly where it stopped» if the model and provider do not support that kind of continuation at the computation level. A more honest interface message is simple: «Continue from the saved state.» It describes what the system actually controls.

Test cancellation under races, not just in a polished demo

A test where the user cancels a slow generation and sees a stopped cursor proves almost nothing. You need scenarios in which events arrive in an inconvenient order.

At minimum, test these cases:

1. The client sent cancel, but the last message.delta was already on the network. The interface must not publish it retroactively.
2. The worker completed a tool between recording cancel_requested and reading the flag. The next run must see the exact operation status.
3. Two tabs resume one checkpoint. One gets a new version, while the other gets a conflict or creates an explicit branch.
4. The client lost its connection but did not cancel the run. On return, it must see the current status instead of automatically treating the work as cancelled.
5. The worker crashed after publishing text but before the final run status. Recovery must find the checkpoint and decide whether a retry is needed, rather than repeating the whole conversation.

For each case, write an invariant check, not just an HTTP-status check. For example: «The history cannot contain a published message without a checkpoint», «one idempotency key creates no more than one external action», «a continuation always references an existing checkpoint», and «a terminal cancelled run publishes no new messages».

Durable workflows have an unpleasant but useful discipline: resuming must not depend on the exact position inside an arbitrary function. LangGraph documentation describes saving checkpoints after steps and restoring the results of completed tasks, not magically returning the processor to a line of code. When work is split into explicit transitions, cancellation becomes an ordinary process state rather than an exception hidden in try/except.

Start with one change that forces the system to tell the truth: add the state version to response publication and require it in resume requests. You will quickly see where you are storing plain text, where you have a checkpoint, and where the agent exists only until the connection breaks.