Skip to content

Let a failed report be regenerated without a rerun - #214

Merged
jserv merged 1 commit into
mainfrom
report
Oct 3, 2026
Merged

jserv merged 1 commit into
mainfrom
report

Conversation

@jserv

@jserv jserv commented Oct 2, 2026 •

Copy link
Copy Markdown
Contributor

An interview that reached its end could still lose every score to a Gemini 503 or the report deadline, and transport retries were already exhausted by then, so the only way back was to sit the whole interview again. The agent now publishes that failure as a provisional report, keeps the frozen assessment in process memory for five minutes, and lets the connected candidate request one regeneration after a 30-second cooldown (60 for quota, and a missed deadline checks the key rotation too). The regeneration reuses the frozen prompt and state with its own five-call pool, so final reporting stays within ten HTTP calls per interview. The agent answers each request on the control topic as accepted, early or closed, and the page times the regeneration from its request or the acceptance rather than from its own clock. Rejected credentials, schema failures, salvaged reports and a candidate who already left get no window. docs/provider-cost-and-degradation.md states the new retention, call bound, LiveKit minutes and log lines.

The page saves the provisional failure on arrival under one report id per interview, and the final outcome replaces it in local history and in the account, whose saves for one id now land in order, so a crashed or closed tab keeps the failure rather than nothing. A fixed local room whose agent is finalizing now refuses a new start as reason=finalizing instead of at_capacity; this includes a reload during an ordinary ending.

Verified with ./scripts/test.sh (green; the Chromium/Playwright lanes, cargo-audit and shellcheck skipped locally for missing tools), including an end-to-end test that drives five 503s from a local HTTP server through the offer, an accepted retry and a complete regenerated report on identical prompts in six calls. cargo mutants --in-diff leaves no survivors outside the documented .cargo/mutants.toml entries for code that only runs against a live LiveKit room. Not exercised against live LiveKit and Gemini credentials. Reload and process-restart recovery remain out of scope, as the doc says.

Closes #97


Summary by cubic

Lets a candidate regenerate a report that failed due to a Gemini 503 or the report deadline, without rerunning the interview.

New Features

  • The agent publishes the failure as a provisional report, keeps the frozen assessment for five minutes, and answers one retry request after a 30-second cooldown (60 for quota); a usable backup key waits only the 30-second floor, a sole key waits the cooldown, and an exhausted rotation waits until its earliest key returns, rounded up to the whole second the page is told. A retry is checked against the keys again on arrival and answered early if they went back on quota since the offer.
  • A rejoin within 30 seconds under the same identity no longer counts as leaving: the accepted regeneration keeps running and its report goes out once the candidate is present. Rejected credentials, schema failures, salvaged reports, and a candidate who left get no retry.
  • Regeneration uses its own five-call pool, keeping final reporting within ten Gemini HTTP calls per interview.

Bug Fixes

  • The page saves the provisional failure immediately, and the final outcome replaces it under the same report id; account saves for one id are now ordered to avoid a race.
  • A provisional packet that parses to null shows the incomplete report instead of the failure overlay, and an early retry answer that lands after the offer expired no longer reopens the recovery window.
  • Recovery restores the ending overlay's title, spinner, and leave label when it ends, and a fixed local room whose agent is finalizing refuses a new start as finalizing instead of at_capacity.
  • The Live session closes after the report is published, or right after the provisional report when a recovery window opens, instead of holding the common ending for the close.

Closes #97.

Written for commit d3afaec. Summary will update on new commits.

Review in cubic

@jserv
jserv requested review from ColtenOuO and alanhc October 2, 2026 04:43
cubic-dev-ai[bot]

This comment was marked as resolved.

Comment thread src/livekit/report.rs Outdated
Comment thread src/livekit/report.rs Outdated
Comment thread web/interview.js
Comment thread src/gemini.rs
Comment thread src/livekit.rs Outdated
cubic-dev-ai[bot]

This comment was marked as resolved.

An interview that reached its end could still lose every score to a
Gemini 503 or the report deadline, and the only way back was to sit the
whole interview again. The agent now publishes that failure as a
provisional report, keeps the frozen assessment in memory for five
minutes, and lets the connected candidate ask once for a regeneration
with its own five-call pool, so final reporting stays within ten HTTP
calls. The wait is 30 seconds, or until the key rotation can answer,
and a retry is checked against the keys again when it arrives. The
agent answers each request on the control topic as accepted, early or
closed, and a candidate who drops has 30 seconds to rejoin before the
wait or a regeneration under way is given up. Rejected credentials,
schema failures, salvaged reports and a candidate already gone get no
window, and the Live session closes after the report as before.

The page saves the provisional failure on arrival under one report id
per interview, and the final outcome replaces it in local history and
in the account, whose saves for one id now land in order, so a crashed
or closed tab keeps the failure rather than nothing. A fixed local room
whose agent is finalizing refuses a new start as finalizing instead of
claiming the server is full.

Closes #97
@jserv
jserv merged commit e365410 into main Oct 3, 2026
8 checks passed
@jserv
jserv deleted the report branch October 3, 2026 02:36
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Report generation fails with Gemini 503 after interview completion

2 participants