Skip to content

Rebuild a resumed session that keeps failing - #213

Merged
jserv merged 2 commits into
mainfrom
resume-rebuild
Oct 2, 2026
Merged

jserv merged 2 commits into
mainfrom
resume-rebuild

Conversation

@jserv

@jserv jserv commented Oct 2, 2026 •

Copy link
Copy Markdown
Contributor

Gemini Live can close a socket with 1011 and then close every socket resumed from its checkpoint 10 to 25 seconds later, so the interview spent all eight restarts resuming the same failing conversation and ended. A resumed socket that dies before it is healthy and debt-free now makes the next replacement rebuild from local state, and resuming stays off until a reply beyond the recovery briefing, or a socket that lives past a minute owing nothing, proves recovery. The reply to a recovery briefing no longer resets the restart budget, since every replacement asks for one, including when that reply goes through a tool call.

During the same loop the candidate's speech was not transcribed, so the idle timers, stamped only from transcripts, counted a talking candidate as silent and nudged them. Until the open socket has transcribed the candidate, their microphone level above room noise now counts as speech for those timers; once it has, transcripts alone count again, so room noise cannot hold nudges back.

A cold rebuild replays the owed prompt and the editor, so if the 1011 is triggered by that input rather than by the resumed conversation, rebuilding will not help. The log now separates the two cases: Gemini cold rebuild also failed before recovery appears when a cold socket fails too, and each replacement: line carries owed_chars= and editor_chars=. No prompt changes, so the interview contract bundle stays at 26.

Verified with ./scripts/test.sh (green; browser lanes, cargo-audit and shellcheck skipped locally for missing tools) and cargo mutants --in-diff over the Rust changes.

Close #198


Summary by cubic

Fixes resumed Gemini sessions that keep failing after a 1011 close, and stops nudging candidates who are talking but not being transcribed.

  • A resumed socket that dies before it is healthy now makes the next replacement rebuild from local state; resuming stays off until a reply beyond the recovery briefing or a socket that stays up past a minute proves recovery. Health is judged by how long the socket stayed up, not the debt-adjusted age the restart budget uses, so a long-lived socket replaced while owing a reply is not blamed as a failed resumption.
  • The reply to a recovery briefing no longer resets the restart budget, whether the briefing was sent at once or held for an unpause or the end of a thinking hold; a briefing answered with silence or cut off by the candidate ends the briefing's turn too, so the next real reply counts as recovery proof.
  • Until a socket has transcribed the candidate, microphone level above room noise counts as speech for the idle timers; once transcribed, transcripts alone count again. A replacement socket must transcribe the candidate again.
  • The log separates a resumed session that fails before recovery from a cold rebuild that also fails, and each replacement: line now carries owed_chars= and editor_chars=; a checkpoint is named only when resumption is actually used.

Migration

  • No prompt changes; the interview contract bundle stays at 26.

Closes #198.

Written for commit 7742a47. Summary will update on new commits.

Review in cubic

cubic-dev-ai[bot]

This comment was marked as resolved.

@jserv
jserv requested review from ColtenOuO and alanhc October 2, 2026 04:18
Gemini Live can close a socket with 1011 and then close every socket
resumed from its checkpoint 10 to 25 seconds later, so the interview
spent all eight restarts resuming the same failing conversation and
ended. A resumed socket that dies before it is healthy and debt-free
now makes the next replacement rebuild from local state, and resuming
stays off until a reply beyond the recovery briefing or a healthy
socket proves recovery. The briefing's own answer no longer resets the
restart budget, since every replacement asks for one. While no socket
transcribes the candidate, a candidate who kept talking was counted
silent and nudged, so until a socket has transcribed them their
microphone level also counts as speech for the idle timers. The log
now says when a cold rebuild fails too, and the size of what each
replacement was handed, which tells a failing checkpoint from input
that fails on any socket.

Close #198
Comment thread src/livekit.rs Outdated
Comment thread src/livekit.rs Outdated
Resumption was suppressed by the debt-adjusted age the restart budget
uses, so a resumed socket that ran for minutes and was replaced while
owing a reply lost its handle, and the log blamed the checkpoint. The
resume decision now uses how long the socket stayed up. A recovery
briefing held through a pause or a thinking hold was answered after
the unpause or the hold's end, and that answer reset the budget, so a
held briefing now marks its reply the way an immediate one does.
@jserv
jserv merged commit cef474a into main Oct 2, 2026
6 checks passed
@jserv
jserv deleted the resume-rebuild branch October 2, 2026 10:05
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Gemini Live gets stuck in a 1011 resume loop and the interview ends itself

2 participants