Repository navigation
Description
TanStack AI version
@tanstack/ai 1.1.0, @tanstack/ai-sandbox 1.1.0, @tanstack/ai-client 1.1.0, @tanstack/ai-claude-code 1.0.1, @tanstack/ai-sandbox-local-process 1.0.0
Framework/Library version
Node.js 26.13.0 and 24.22.2. No UI framework: the server and the clients run in one Node script.
Describe the bug and the steps to reproduce it
When a client reconnects to a durable sandbox run while the agent is in a tool call that lasts more than 5 seconds, the rest of the run is written to the run log twice. The reconnected client receives every later event twice, and the text deltas interleave:
Running the long command.The The long long command command finished. finished. Each Each word word of of this this reply reply is is one one delta.delta.
Expected: every event arrives once, as it does when nobody reconnects.
Steps. The repro does all of them with pnpm install && pnpm start. It needs no API key.
- A POST route starts a durable run as
docs/sandbox/takeover.mdshows:chat()withclaudeCodeText,withLocks(locks), andwithSandbox(sandbox, { runs, instances, durability: { adapter } }), returned withtoServerSentEventsResponse(stream, { durability: { adapter } }). The sandbox islocalProcessSandbox(). The log ismemoryStream. The stores areInMemoryRunStore,InMemorySandboxInstanceStore, andInMemoryLockStore. - A GET route returns
resumeServerSentEventsResponsewithsandboxRunDriver({ request, runs, locks, durability, drive }).drivecallschat()again withattach: true. - The agent makes one Bash call that takes 15 seconds, then streams a 12-word answer. To run without a model,
claudeExecutablepoints atfake-claude.mjs, which prints Claude Code stream-json. - At 3 seconds, a
ChatClientcallsstop()with nocancelUrl. The connection closes and the run is not cancelled. - At 3.5 seconds, a second client finds the active run and calls
fetchServerSentEvents(url).joinRun(runId).
Output:
[ 15 ms] locks.withLock("sandbox:80f0ccca17217a27")
[ 3004 ms] tab 1: disconnects (no cancel)
[ 3511 ms] tab 2: joinRun("run-1793812048416-e9jvxh")
[ 3513 ms] locks.withLock("run-driver:run-1793812048416-e9jvxh")
[ 8514 ms] takeover: drive() starts
[ 8515 ms] locks.withLock("sandbox:80f0ccca17217a27")
...
Tab 2 received:
TOOL_CALL_RESULT 2 (expected 1)
TEXT_MESSAGE_CONTENT 25 (expected 13: 1 before the tool call, 12 after)
RUN_FINISHED 1 (expected 1)
The run log holds:
TOOL_CALL_RESULT 2 (expected 1)
TEXT_MESSAGE_CONTENT 25 (expected 13: 1 before the tool call, 12 after)
RUN_FINISHED 2 (expected 1)
Appends to the run log, by producer:
POST producer: 37 chunks, TOOL_CALL_RESULT 1, TEXT_MESSAGE_CONTENT 13, RUN_FINISHED 1
takeover driver: 18 chunks, TOOL_CALL_RESULT 1, TEXT_MESSAGE_CONTENT 12, RUN_FINISHED 1
RESULT: DUPLICATED
The events were duplicated in 7 of 7 runs, on Node 26 and Node 24. With a 4-second tool call (TOOL_MS=4000 DISCONNECT_AT_MS=1000 pnpm start), every event arrived once in 3 of 3 runs. We first saw this with the real Claude Code CLI in a Vercel sandbox, a Redis log, and a 45-second Bash command.
Suspected cause (not confirmed). After the disconnect, toServerSentEventsResponse keeps draining the POST run into the log, as it should for a detached run. That producer never takes the run-driver:<runId> lock or a driverEpoch. The lock lines above show that only the takeover takes it. So fenceDurability cannot stop the POST producer, and the only remaining guard is awaitLogQuiescence (DEFAULT_FENCE_QUIET_MS = 5_000). A tool call that keeps the log quiet for more than 5 seconds passes that guard, and then both producers translate the same journal and append the same events. For this reason, I expect a distributed lock store to give the same result. The e2e test "a real disconnect, then an attach, continues the stream" simulates the disconnect by aborting the POST run's controller. That abort stops the POST producer, so the test may not cover this path.
Your Minimal, Reproducible Example - (Sandbox Highly Recommended)
https://github.com/adelrodriguez/tanstack-ai-repros/tree/main/takeover-duplicates-stream
Screenshots or Videos (Optional)
No response
Do you intend to try to help solve this bug with your own PR?
Maybe, I'll investigate and start debugging
Terms & Code of Conduct
- I agree to follow this project's Code of Conduct
- I understand that if my bug cannot be reliable reproduced in a debuggable environment, it will probably not be fixed and this issue may even be closed.
Activity
fix(ai-sandbox): hold the driver claim for a fresh durable run