Skip to content

fix(execution): keep terminal reconnect off runs a Sim run tool owns - #7382

Merged
icecrasher321 merged 6 commits into
stagingfrom
fix/reconnect-skips-live-run-tool
Sep 2, 2026
Merged

fix(execution): keep terminal reconnect off runs a Sim run tool owns#7382
icecrasher321 merged 6 commits into
stagingfrom
fix/reconnect-skips-live-run-tool

Conversation

@icecrasher321

@icecrasher321 icecrasher321 commented Sep 2, 2026

Copy link
Copy Markdown
Collaborator

Summary

Running a workflow from Chat on staging produced a "Run Error: Execution state is no longer available after reconnect" entry (0ms, last in the group) on a run whose blocks all succeeded. The entry is synthetic: the terminal's reconnect effect wrote it after a reconnect GET .../executions/{id}/stream returned 404. Tracing it end to end surfaced three gaps in how the Chat run tool and the terminal reconnect flow share a run, and this PR closes all three at the level they live.

Root cause of the reported error

  • doExecuteRunTool (apps/sim/lib/copilot/tools/client/run-tool-execution.ts) sets currentExecutionId and writes the sessionStorage execution pointer before sending the execute POST. It is the only run path that writes the pointer before the server has created the run buffer; the manual path and executeWorkflowWithFullLogging write it on the X-Execution-Id header.
  • Starting the run adds and activates the workflow tab in Chat, which mounts useWorkflowExecution (embedded workflow.tsx and EmbeddedWorkflowActions), and setActiveWorkflow flips activeWorkflowId after the workflow fetch. Both re-run the hook's reconnect effect mid-run.
  • The effect's claim check only knows "pointer present and the current execution id matches", which is exactly the state a live run tool creates. cancelExecute is a no-op for the tool's raw fetch, so the effect opened a reconnect GET in parallel with the live POST.
  • That GET races the POST's initializeExecutionStreamMeta; when it reads first, the stream route answers 404, which the effect treats as non-retryable: it logs the Run Error with the live run's execution id, sets isExecuting false, nulls the current execution id, and clears the pointer the tool keeps for reload recovery. The POST headers then restore the id and the blocks stream in normally, which is why the run still looked successful. The error entry gets executionOrder: MAX_SAFE_INTEGER, so it always sorts last.

Changes

1. The reconnect effect defers to a live run tool, and the tool hands an interrupted run back (commit 1)

  • isRunToolActiveForWorkflow(workflowId) exposes the run tool's existing per-workflow ownership map. The reconnect effect returns early while it is set, before loading the pointer and without clearing it.
  • When the tool gives up an interrupted run it notifies subscribeToRunToolRelease subscribers after releasing ownership; the hook subscribes and bumps reconnectAttemptNonce, so the terminal re-attaches from the last persisted event, matching what the manual path already does on interruption. Runs the tool observed to completion never notify, so a failed completion report still leaves the pointer for bindRunToolToExecution to re-report after a reload.

2. One stream-interruption classifier for every live stream (commit 2)

  • Only useExecutionStream.execute wrapped a transport failure as SSEStreamInterruptedError; executeWorkflowWithFullLogging rethrew the raw TypeError. So on the Chat path a mid-run network drop took the generic branch: the tool reported error, the confirm route marked the row failed, and the pointer was cleared while the execute route kept running the workflow (the streaming branch never aborts on client disconnect).
  • toStreamInterruptedError is now the single exported classifier, used by both hook execute paths (pure extraction, no behaviour change) and by the shared executor's post-acknowledgement catch. The run tool therefore reaches its recoverable branch: reports background (truthful, the server continues), keeps the pointer, and releases the run to the terminal reconnect via change 1.
  • Review rounds hardened the classifier in place: it matches the browsers' transport messages as patterns (so Firefox's NetworkError when attempting to fetch resource. counts), tolerates nullish or non-object rejections, and never classifies the stream layer's own typed errors (ExecutionStreamHttpError, SSEEventHandlerError) by their message text.

3. Async launches no longer write the terminal pointer (commit 2)

  • The async path wrote the pointer only so bindRunToolToExecution would find something after a reload, but an async run has no reconnectable stream, so any reconnect against that pointer 404'd into the same synthetic Run Error.
  • The tab-local pending completion report already carries the execution id. Async launches now write only that, and recovery answers from the pending report first, falling back to the pointer only for a live run this tab was observing. The legacy clearExecutionPointerAfterReport flag is still honoured so pointers older clients left behind get cleaned up.

Traced end to end across both commits: manual runs and run tools cannot overlap on a workflow, reload recovery is unchanged for live runs, async runs were never stoppable from the terminal so nothing loses the pointer, no import cycle, nothing new in the editor bundle. Three independent adversarial reviews over the diffs; their findings that survived are the two accepted notes below.

Tests

  • Hook: the 404 path on an unowned pointer still logs the Run Error (proves the guard test is not vacuous); an owned pointer skips reconnect, logs nothing, and keeps store and pointer intact; a release for the active workflow reconnects from the pointer's lastEventId, one for another workflow does not, and unmount unsubscribes.
  • Run tool: ownership is set before the pointer write and held exactly while the run is in flight; both interruption kinds release only after ownership and store teardown, keep the pointer, and send a background confirm; a completed run whose report fails does not release; async launches never touch the pointer and recover from the pending report alone; a legacy async pointer is cleaned up after its pending report is re-sent.
  • Shared executor: a transport drop after the server acknowledged the run rejects with SSEStreamInterruptedError carrying the server execution id; an AbortError, the run tool's string abort reason, and a non-transport error are rethrown untouched.
  • Mutation-tested: removing the guard, the nonce bump, the notify, the streamInterrupted condition, the ownership write, the executor wrap, the classifier, the pointer-free async launch, or the pending-first recovery each fails exactly the test written for it.
  • bun run test on the four files plus chat.test.tsx, bun run type-check (all packages), biome on every changed path, and bun run check:audits (44/44) pass.

Accepted notes from review

  • A server-initiated stream teardown (buffer write failure, which aborts the run and errors the stream) now reads as a transport drop on the Chat path and is reported as background; the same classification the manual path already makes, and the confirm route resolves against the trusted execution on any later confirm.
  • A tab running the previous client with a stuck async pointer that reloads onto this build can log the synthetic Run Error once before bindRunToolToExecution cleans the pointer up, exactly as the previous client did to itself.

🤖 Generated with Claude Code

@vercel

vercel Bot commented Sep 2, 2026

Copy link
Copy Markdown

The latest updates on your projects. Learn more about Vercel for GitHub.

1 Skipped Deployment
Project Deployment Actions Updated
docs Skipped Skipped Sep 2, 2026 2:52am UTC

Request Review

@greptile-apps

greptile-apps Bot commented Sep 2, 2026

Copy link
Copy Markdown
Contributor

Greptile Summary

The PR coordinates Chat run-tool ownership with terminal reconnection, standardizes transport-interruption classification, and avoids creating terminal pointers for asynchronous launches.

  • Defers reconnect while a live run tool owns the workflow and retries after interrupted ownership is released.
  • Preserves acknowledged executions when their streams suffer browser transport failures.
  • Recovers asynchronous completion reports without treating asynchronous launches as reconnectable streams.

Confidence Score: 5/5

The PR appears safe to merge.

No blocking failure remains.

Important Files Changed

Filename Overview
apps/sim/lib/copilot/tools/client/run-tool-execution.ts Adds workflow-scoped run ownership and release notifications while separating asynchronous completion recovery from live execution pointers.
apps/sim/app/workspace/[workspaceId]/w/[workflowId]/hooks/use-workflow-execution.ts Defers terminal reconnection to an active run tool and retries when an interrupted run is released.
apps/sim/hooks/use-execution-stream.ts Extracts a shared classifier for browser transport failures that interrupt acknowledged execution streams.
apps/sim/app/workspace/[workspaceId]/w/[workflowId]/utils/workflow-execution-utils.ts Preserves acknowledged executions for recovery when the shared executor encounters a classified stream interruption.
apps/sim/lib/copilot/tools/client/run-tool-execution.test.ts Replaces the previously reported untyped callback parameters and verifies abort-signal, ownership, release, and asynchronous recovery behavior.

Reviews (5): Last reviewed commit: "fix(execution): never classify the strea..." | Re-trigger Greptile

Comment thread apps/sim/lib/copilot/tools/client/run-tool-execution.test.ts

@cubic-dev-ai cubic-dev-ai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

All reported issues were addressed across 4 files

Heads up: you’re close to your included review allowance. Set a flex budget so reviews don’t pause.

Fix all with cubic | Re-trigger cubic

icecrasher321 and others added 3 commits September 1, 2026 19:21
The terminal's reconnect effect treated "execution pointer present and the
current execution id matches" as an orphaned run and claimed it. A Chat run
tool creates exactly that state before the server has acknowledged the run,
and opening the workflow tab in Chat re-runs the effect mid-run, so the
reconnect GET raced the execute POST's buffer init, got a 404, and logged
"Execution state is no longer available after reconnect" as a Run Error on a
run that succeeded. It also tore down the live run's store state and cleared
the pointer the tool keeps for reload recovery.

The run tool now exposes its ownership (isRunToolActiveForWorkflow) and the
reconnect effect skips a workflow whose run it owns, leaving the pointer in
place. When the tool gives up an interrupted run it notifies
subscribeToRunToolRelease subscribers and the hook re-arms its reconnect, so
the terminal re-attaches from the last persisted event the way the manual
run path already does on interruption. Runs the tool observed to completion
never notify, so a failed completion report still leaves the pointer for
bindRunToolToExecution to re-report after a reload.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
…iting a terminal pointer

Only useExecutionStream.execute wrapped a transport failure as
SSEStreamInterruptedError; executeWorkflowWithFullLogging rethrew the raw
TypeError, so a mid-run network drop on the Chat run-tool path took the
generic branch: the tool reported "error" to Sim, the confirm route marked
the row failed, and the pointer was cleared while the server kept running
the workflow. The classifier is now one exported helper
(toStreamInterruptedError) used by both execute paths and by the shared
executor's post-acknowledgement catch, so the run tool reaches its
recoverable branch, reports "background", keeps the pointer, and releases
the run to the terminal reconnect.

Async launches wrote the terminal execution pointer only so
bindRunToolToExecution would find something after a reload, but an async
run has no reconnectable stream, so any reconnect against that pointer
404'd into the same synthetic Run Error. The tab-local pending completion
report already carries the execution id, so async launches no longer touch
the pointer and recovery answers from the pending report first, falling
back to the pointer only for a live run this tab was observing. The legacy
clearExecutionPointerAfterReport flag is still honoured for pointers older
clients left behind.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
…s contract

Greptile flagged the new mock's `options: any`; the sibling abort test had the
same shape. Export WorkflowExecutionOptions from the shared executor and use it
in both, with a helper that fails the test if the run tool ever stops passing
an abort signal.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
@icecrasher321
icecrasher321 force-pushed the fix/reconnect-skips-live-run-tool branch from 029a959 to f431355 Compare September 2, 2026 02:22
@icecrasher321

Copy link
Copy Markdown
Collaborator Author

@greptile

@icecrasher321

Copy link
Copy Markdown
Collaborator Author

@cubic-dev-ai review this PR

@cubic-dev-ai

cubic-dev-ai Bot commented Sep 2, 2026

Copy link
Copy Markdown
Contributor

@cubic-dev-ai review this PR

@icecrasher321 I have started the AI code review. It will take a few minutes to complete.

@cubic-dev-ai cubic-dev-ai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

All reported issues were addressed across 7 files

Reply with feedback, questions, or to request a fix.

Fix all with cubic | Re-trigger cubic

Comment thread apps/sim/hooks/use-execution-stream.ts
The transport-failure matcher only knew Chrome's "network error" with a
space, so Firefox's "NetworkError when attempting to fetch resource."
fell through as a plain failure. Now that every live stream shares this
classifier, match the browsers' known messages as patterns and cover each
form in the executor test.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
@icecrasher321

Copy link
Copy Markdown
Collaborator Author

@greptile

@icecrasher321

Copy link
Copy Markdown
Collaborator Author

@cubic-dev-ai review this PR

@cubic-dev-ai

cubic-dev-ai Bot commented Sep 2, 2026

Copy link
Copy Markdown
Contributor

@cubic-dev-ai review this PR

@icecrasher321 I have started the AI code review. It will take a few minutes to complete.

@cubic-dev-ai cubic-dev-ai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

All reported issues were addressed across 7 files

Reply with feedback, questions, or to request a fix.

Fix all with cubic | Re-trigger cubic

Comment thread apps/sim/hooks/use-execution-stream.ts
…ections

isClientDisconnectError read error.name unguarded, so a stream that rejected
with null or undefined would throw inside the catch and mask the original
failure. Both predicates now take unknown and bail on non-object values; the
executor test covers a nullish body-reader rejection.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
@icecrasher321

Copy link
Copy Markdown
Collaborator Author

@greptile

@icecrasher321

Copy link
Copy Markdown
Collaborator Author

@cubic-dev-ai review this PR

@cubic-dev-ai

cubic-dev-ai Bot commented Sep 2, 2026

Copy link
Copy Markdown
Contributor

@cubic-dev-ai review this PR

@icecrasher321 I have started the AI code review. It will take a few minutes to complete.

@cubic-dev-ai cubic-dev-ai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

All reported issues were addressed across 7 files

Reply with feedback, questions, or to request a fix.

Fix all with cubic | Re-trigger cubic

Comment thread apps/sim/hooks/use-execution-stream.ts
…port drops

An ExecutionStreamHttpError or SSEEventHandlerError whose message happened to
contain a browser transport phrase ("Failed to fetch workflow state") would
have been re-wrapped as a stream interruption, losing the HTTP status and
taking the recovery path for a run that never started. The predicate now
excludes the stream layer's typed errors before looking at message text.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
@icecrasher321

Copy link
Copy Markdown
Collaborator Author

@greptile

@icecrasher321

Copy link
Copy Markdown
Collaborator Author

@cubic-dev-ai review this PR

@cubic-dev-ai

cubic-dev-ai Bot commented Sep 2, 2026

Copy link
Copy Markdown
Contributor

@cubic-dev-ai review this PR

@icecrasher321 I have started the AI code review. It will take a few minutes to complete.

@cubic-dev-ai cubic-dev-ai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No issues found across 7 files

Confidence score: 5/5

  • Automated review surfaced no issues in the provided summaries.
  • No files require special attention.

Re-trigger cubic

@icecrasher321
icecrasher321 merged commit 9a6a8b1 into staging Sep 2, 2026
30 checks passed
@icecrasher321
icecrasher321 deleted the fix/reconnect-skips-live-run-tool branch September 2, 2026 03:00
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant