Skip to content

fix(studio): fail a chat turn whose model stream goes idle - #850

Merged
anandpant merged 1 commit into
mainfrom
fix/phase3-chat-idle-bound
Nov 17, 2026
Merged

anandpant merged 1 commit into
mainfrom
fix/phase3-chat-idle-bound

Conversation

@anandpant

@anandpant anandpant commented Nov 17, 2026 •

Copy link
Copy Markdown
Collaborator

A Studio model response that stopped sending events held its chat request open forever.

The Studio OpenAI client override in apps/studio/src/features/chat/model.server.ts now limits two waits to 60 seconds each: the wait for response headers (Effect.timeoutOrElse) and each gap between provider events (Stream.timeoutOrElse, applied before preserveOutputLimit). A stall fails with an AiError. The existing AG-UI lifecycle turns that into RUN_ERROR, and the browser already shows RUN_ERROR as an error. Interrupting the stream aborts the gateway request.

Tool execution runs outside the provider stream, so tools are not timed. Pre-810 chat (2b4ad29d) had no idle bound to match, so 60 seconds is new policy. Studio uses reasoning effort none, so we expect a healthy stream to send events every few seconds. That is a policy expectation, not a measured guarantee. A timeout that fires too early kills a valid turn, so the bound leans long.

Proof

  • The boundary test in agent.server.test.ts drives the real handler and gateway with TestClock. It stalls before headers in one run and after events in another. Each run ends in RUN_ERROR with no RUN_FINISHED, the gateway aborted and exactly one gateway call. In the second run, two 59-second gaps between events (118 seconds in total) do not time out.

  • Each of these mutations makes the test fail: removing the stream timeout, removing the headers timeout, replacing the idle bound with a total deadline, or using the silent Stream.timeout.

  • Studio tests 218/218. pnpm ci:local --no-signoff against origin/main passes in 2m47s.

  • Signoff: pnpm ci:local on the pushed head dd8260f5 passed and posted local-ci.

  • Independent Sol review: ACCEPT on dd8260f5, no findings. 28/28 focused tests and 3 independent mutations; request-owned telemetry is kept in both stall cases.

  • Local split browser proof: PASS. This is not full end-to-end timeout proof. The real handler, gateway transport and OpenAI decoder produced the stall SSE: RUN_ERROR before headers, two 59-second gaps kept alive, then RUN_ERROR, plus a healthy turn ending in RUN_FINISHED. The built Studio page replayed those captures and showed "The agent run failed." with Try again. Retrying started a new run that recovered with a rendered reply. Console and page errors were empty. No paid, model or remote calls were made. Browser timing, real provider or Worker cancellation, preview and production were not tested.

A model response that stopped sending events held its chat request open
forever. The Studio OpenAI client now bounds the wait for response
headers and each gap between provider events at 60 seconds with Effect's
native timeouts. A stall fails the stream with an AiError, which the
existing AG-UI lifecycle turns into RUN_ERROR, and interrupting the
stream aborts the gateway request. Tool execution runs outside the
provider stream and is not timed.

Pre-810 chat had no idle bound to match. The boundary test drives the
real handler with TestClock: a stall before headers and a stall after
active events both end in RUN_ERROR with the gateway cancelled and no
retry, and events arriving just inside the bound keep the turn alive.
@anandpant
anandpant marked this pull request as ready for review November 17, 2026 00:11
@anandpant
anandpant merged commit a70feeb into main Nov 17, 2026
10 checks passed
@anandpant
anandpant deleted the fix/phase3-chat-idle-bound branch November 17, 2026 00:11
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant