Recover an interviewer that stops answering

README.md

@@ -211,6 +211,7 @@ The common ones:
 | `GEMINI_LIVE_MODEL` | `gemini-3.1-flash-live-preview` | Realtime interviewer model |
 | `GEMINI_REPORT_MODEL` | `gemini-3.1-flash-lite` | Report model |
 | `CODETRIAL_MAX_INTERIM_REVIEWS` | `6` | Quiet-pause report-model reviews per interview; `0` disables them and `72` is the maximum |
+| `CODETRIAL_GEMINI_REPLY_TIMEOUT_S` | `45` | Seconds an owed interviewer reply may go without output before the Live socket is replaced (20–120); see [degradation controls](docs/provider-cost-and-degradation.md) |
 | `CODETRIAL_GEMINI_CANDIDATE_VIDEO_ENABLED` | `false` | Forward candidate video to Gemini, one low-resolution frame in five seconds |
 | `CODETRIAL_COMPILER_EXPLORER_ENABLED` | `true` | Enable remote C, C++, and Java runs |
 | `CODETRIAL_MAX_CONCURRENT_INTERVIEWS` | `16` | Interviews one `web` process hosts agents for |

config/codetrial.env.example

@@ -8,6 +8,10 @@ GEMINI_REPORT_MODEL=gemini-3.1-flash-lite
 # Reviews quiet candidate stretches with the report model. 0 reserves that
 # quota for final reports; the default is 6 and the maximum is 72.
 # CODETRIAL_MAX_INTERIM_REVIEWS=6
+# Seconds an owed interviewer reply may go without output before the Live
+# socket is replaced. Raise it if `timing:` lines show slow replies that do
+# arrive; the default is 45, bounded to 20-120.
+# CODETRIAL_GEMINI_REPLY_TIMEOUT_S=45
 GEMINI_VOICE=Puck
 CODETRIAL_ROOM_PREFIX=interview
 CODETRIAL_DURATION_MIN=45

docs/interview-contract-versions.md

@@ -8,10 +8,11 @@ can select it.
 
 ## The active bundle
 
-Bundle 25: live prompt 17, report prompt 15, rubric 1, report schema 2.
+Bundle 26: live prompt 18, report prompt 15, rubric 1, report schema 2.
 
 | Bundle | Introduced |
 |---|---|
+| 26 | When a lost connection leaves a reply owed, the request for it is appended to whatever the interviewer is sent next: the cold briefing of a replacement that cannot resume, now including a reply owed for the candidate's own turn, and on unpause the cold briefing as well as the resume line. The briefings themselves are unchanged. |
 | 25 | Candidates can keep the floor while thinking, reclaim it during a reply, and yield it early. Explicit spoken requests for thinking time in English, including one that follows an answer in the same sentence or is asked as a question, suppress generated replies and automatic nudges until the candidate speaks again or chooses to continue. A hold ends on its own at the five-minute warning, at the round transition, and after two silent minutes with one brief check-in; the interviewer is told that anything it said during the hold was not heard. A Continue within ten seconds of the last one releases the hold without a reply of its own. Thinking keeps editor, microphone and test evidence live, gives the interviewer test runs and edits as context it does not answer, and never extends the deadline. The default endpointing window is three seconds, and the page shows it filling while the candidate is silent; yielding ends the audio stream so the interviewer replies without waiting it out. |
 | 24 | The Live main instructions drop repeated explanations and illustrative examples and keep every timer, round, evidence-source and hint restriction. The greeting answers only the platform's startup request, and missing history, a compression or a tool result is not a new interview. `end_interview` is called silently, before any acknowledgment or goodbye, and the platform supplies the closing. A cut `read_editor` page or a checkpoint excerpt does not show the whole buffer, so an implementation or technique is not called absent before the named lines are read. The `read_editor` description asks for only the code the current question needs that nothing has shown, from a known relevant line rather than a refill of the whole editor. The greeting no longer repeats the exercise's title and brief, which THE EXERCISE already carries and the greeting now points at; the framework headers drop a scoring premise the disclosure rule already covers; test-run reactions and the earlier-steps reminder state their rule once, more briefly; and the `end_interview` description no longer restates the instruction it sits beside. With a configured compression window, a silent checkpoint rebuilt from local state follows a detected cut: the chosen language, the current round, the evidence, a bounded transcript that keeps a long behavioral round's opening, a bounded test report and, in the coding round, the editor's opening and ending. Its next step applies to the next candidate input, not to the checkpoint itself. Omission alone does not close a behavioral round, repeat its question or establish that its follow-up is unused, and a refusal or request to finish supplies no STAR evidence. Under the same window, editor, hint and evidence tool answers carry the latest unanswered candidate utterance as quoted historical data, never as a new turn. |
 | 23 | A candidate who hides the worked examples in the preflight sends `hideExamples` with the token request, and the live prompt then says no examples are on their screen: the interviewer never points them at one, says a clarification or hint clue that mentions an example with a case they proposed or one of its own, and in the Example step asks for their ordinary and boundary cases before offering a small example once they have tried or are stuck. A session that does not hide them gets the live prompt unchanged. |

docs/provider-cost-and-degradation.md

@@ -17,13 +17,52 @@ server issued a handle; if the handle is unavailable or refused, the restart is
 logged as degraded and is grounded from the bounded local transcript tail,
 editor, round and evidence state instead.
 
+A candidate turn, required prompt (including the opening greeting), or tool
+continuation that produces no output for 45 seconds replaces the socket even
+without a `GoAway`. `CODETRIAL_GEMINI_REPLY_TIMEOUT_S` sets that interval
+within 20 to 120 seconds. The `timing:` lines log how long each reply took to
+start; a reply the watchdog replaces is paid for twice, so an endpoint whose
+slow replies do arrive warrants a longer one. A candidate turn is timed from
+the last transcript fragment of it, and a fragment arriving within two seconds
+of a turn that said something is taken as the lagging tail of speech that turn
+already answered, so it owes nothing. The replacement, resumed or cold, carries
+the unanswered reply across and publishes the same reconnecting state. A reply
+cut off mid-generation is owed too, and the replacement is told not to repeat
+what was already said. Optional editor reviews accept silence; queued audio and
+a paused interview do not trigger this watchdog. A generation that stops
+producing output for the same interval also recovers. Periodic nudges wait
+while a reply is owed rather than replacing its debt.
+
+A close the interviewer asked for is not recovered either: it waits on the
+tool acknowledgement, and if that never comes, or starts and then stops, the
+interview closes once the acknowledgement has been silent for 20 seconds. A
+close is held across a pause and acted on after the resume. A generation or
+tool continuation silent for 20 seconds when the candidate pauses is not
+waited on after the resume, so the reply to the resume is heard rather than
+discarded, and if that generation does end later its ending is not taken as
+the answer to the resume.
+
+A turn Gemini completes with no output settles what it owed. The socket has
+answered, and the model may choose silence; the idle nudges, not the watchdog,
+respond to a silence that goes on. It is logged as a deliberate silence, so a
+report of the interviewer going quiet can be told apart from a stalled socket.
+
 `GEMINI_RESTART_LIMIT` bounds a failing endpoint rather than a long interview.
-It allows 8 opens in a row, and any socket that lived past a minute clears the
-run. A project that cannot pay is not an endpoint that may recover: a 402, or a
-close reason saying the prepaid credit is depleted or billing is not enabled,
-takes that key off both the Live and the report surface and is retried only on
-another configured key. With none left the interview ends at once instead of
-spending the remaining opens, and its summary line says `outcome=billing`.
+It allows 8 opens in a row. A completed turn with output clears the run, and so
+does replacing a socket that lived past a minute and owed nothing. A socket
+replaced while it owed a reply never counts as healthy, however long it stayed
+connected and whether the watchdog, a `GoAway` or the server closed it, so
+repeated unanswered recovery briefings exhaust the budget. Exhausting it, or
+failing to open a replacement socket for any reason but billing, ends the
+interview with reason `interviewer_unavailable`: no goodbye is asked for, the
+reconnecting notice is withdrawn, and the report is still written from the
+session held so far. A first socket that cannot be opened at all still ends the
+session before it starts, with no report. A project that cannot pay is not an
+endpoint that may recover: a 402, or a close reason saying the prepaid credit is
+depleted or billing is not enabled, takes that key off both the Live and the
+report surface and is retried only on another configured key. With none left the
+interview ends at once instead of spending the remaining opens, and its summary
+line says `outcome=billing`.
 
 Every Live turn is billed on the whole context it runs in, retained audio and
 images included, so what stays in the context costs again on every later turn.
@@ -80,12 +119,14 @@ cross-session response cache.
 
 The browser exposes distinct accessible states for connecting, live,
 reconnecting, offline practice, report generation, incomplete report, and
-retry-ready. Reconnecting preserves the live session and resends current code.
-Offline practice keeps the editor and local tests usable while explicitly
-promising no personalized evaluation. An invalid or exhausted report says that
-no scores or verdict were created. Browser fallback may summarize local test
-progress only for a session that never reached an interviewer, and never
-presents canned feedback as an agent evaluation.
+retry-ready. A reply owed for four seconds with nothing started on it, or one
+that stops producing after its audio runs out, shows the interviewer as thinking
+rather than listening. Reconnecting preserves the live session and resends
+current code. Offline practice keeps the editor and local tests usable while
+explicitly promising no personalized evaluation. An invalid or exhausted report
+says that no scores or verdict were created. Browser fallback may summarize
+local test progress only for a session that never reached an interviewer, and
+never presents canned feedback as an agent evaluation.
 
 ## Operating it
 

src/agent.rs

@@ -64,7 +64,7 @@ pub use prompts::{
     report_system_instruction, resume, resumed_context, rolling_assessment, round_skipped,
     round_started, silence_nudge, spoken_language, test_results_reaction,
     test_runner_unavailable_reaction, test_setup_error_reaction, time_warning,
-    unrecorded_earlier_phases, wrap_up,
+    unrecorded_earlier_phases, with_owed_reply, wrap_up,
 };
 pub(crate) use prompts::{editor_tool_continuity, end_interview_refusal};
 pub(crate) use report::sanitize_report_candidate;
@@ -165,8 +165,8 @@ pub const THINKING_CHECK_IN_S: u64 = 120;
 pub(crate) const THINKING_RELEASE_COOLDOWN: std::time::Duration =
     std::time::Duration::from_secs(10);
 
-pub const INTERVIEW_CONTRACT_BUNDLE_VERSION: u32 = 25;
-pub const LIVE_PROMPT_VERSION: u32 = 17;
+pub const INTERVIEW_CONTRACT_BUNDLE_VERSION: u32 = 26;
+pub const LIVE_PROMPT_VERSION: u32 = 18;
 pub const REPORT_PROMPT_VERSION: u32 = 15;
 pub const RUBRIC_VERSION: u32 = 1;
 pub const REPORT_SCHEMA_VERSION: u32 = 2;
@@ -852,9 +852,10 @@ pub struct RuntimeState {
     pub needs_cold_brief: bool,
     /// A resumed socket replaced a session that owed a reply while the
     /// interview was paused. The reply cannot be asked for then, since it
-    /// would be discarded, so this request for it, owed event included, is
-    /// spoken after the resume line on unpause.
-    pub owed_reply_on_resume: Option<String>,
+    /// would be discarded. Hold the raw event, not its formatted request, so
+    /// recovery cannot nest briefings. Some(None) owes a reply without a known
+    /// event; None owes no reply.
+    pub owed_reply_on_resume: Option<Option<String>>,
     /// Observations a reviewer recorded in the pauses, while the interview was
     /// still running. Held apart from `framework_evidence`, which is the
     /// interviewer's own bookkeeping about which phase happened: these are the

src/agent/prompts.rs

@@ -1427,6 +1427,17 @@ pub fn owed_reply(owed_prompt: Option<&str>) -> String {
     }
 }
 
+/// A briefing or resume line, followed by the request for the reply a lost
+/// connection owed when one is owed. `Some(None)` owes a reply with no event
+/// to name. One join for every path that asks, so the unpause and a cold
+/// replacement cannot phrase the request differently.
+pub fn with_owed_reply(line: String, owed: Option<Option<&str>>) -> String {
+    match owed {
+        Some(owed_prompt) => format!("{line} {}", owed_reply(owed_prompt)),
+        None => line,
+    }
+}
+
 /// Why `end_interview` is refused: coding-only time belongs to the candidate,
 /// and an unfinished round needs only the steps still missing. With Test
 /// recorded, inviting a Run sends a

src/agent/turn_taking.rs

@@ -197,8 +197,11 @@ fn thinking_debt(state: &RuntimeState) -> Vec<String> {
     if state.thinking_unheard_reply {
         prompts.push(THINKING_UNHEARD.to_string());
     }
+
+    // Held as the raw event and phrased only here, so a request that is itself
+    // owed again is never wrapped inside another.
     if let Some(owed) = &state.owed_reply_on_resume {
-        prompts.push(owed.clone());
+        prompts.push(super::owed_reply(owed.as_deref()));
     }
     prompts
 }

src/config.rs

@@ -85,6 +85,17 @@ pub const DEFAULT_MAX_CONCURRENT_INTERVIEWS: usize = 16;
 pub const DEFAULT_MAX_INTERIM_REVIEWS: usize = 6;
 pub const MAX_INTERIM_REVIEWS: usize = 72;
 
+/// How long an owed reply may go without output before the Live socket is
+/// replaced. Issue 95 logged a 39-second reply on a degraded provider that did
+/// arrive, so the default sits above it; a reply the watchdog replaces is paid
+/// for twice. Bounded below by the twenty seconds after which an unanswered
+/// prompt already hands the floor back, since a shorter timeout would replace
+/// sockets a held `GoAway` is still waiting on, and above so that a silent
+/// provider is still recovered while the candidate is waiting for it.
+pub const DEFAULT_GEMINI_REPLY_TIMEOUT_S: u32 = 45;
+pub const MIN_GEMINI_REPLY_TIMEOUT_S: u32 = 20;
+pub const MAX_GEMINI_REPLY_TIMEOUT_S: u32 = 120;
+
 const REQUIRED_KEYS: &[&str] = &[
     "LIVEKIT_URL",
     "LIVEKIT_API_KEY",
@@ -546,6 +557,7 @@ pub struct AgentConfig {
     pub default_duration_min: u32,
     pub gemini_candidate_video_enabled: bool,
     pub max_interim_reviews: usize,
+    pub gemini_reply_timeout_s: u32,
     pub pool: ProviderPool,
 }
 
@@ -736,6 +748,12 @@ pub fn load_from_pairs(
             DEFAULT_MAX_INTERIM_REVIEWS as u32,
         )
         .min(MAX_INTERIM_REVIEWS as u32) as usize,
+        gemini_reply_timeout_s: optional_u32(
+            &values,
+            "CODETRIAL_GEMINI_REPLY_TIMEOUT_S",
+            DEFAULT_GEMINI_REPLY_TIMEOUT_S,
+        )
+        .clamp(MIN_GEMINI_REPLY_TIMEOUT_S, MAX_GEMINI_REPLY_TIMEOUT_S),
         pool,
     })
 }

src/gemini.rs

@@ -1810,6 +1810,13 @@ fn parse_server_message(text: &str) -> ServerMessage {
         return ServerMessage::default();
     };
     let mut events = Vec::new();
+    let interrupted = message
+        .pointer("/serverContent/interrupted")
+        .and_then(Value::as_bool)
+        .unwrap_or(false);
+    let input_transcript = message
+        .pointer("/serverContent/inputTranscription/text")
+        .and_then(Value::as_str);
 
     let resumption_handle = message
         .get("sessionResumptionUpdate")
@@ -1824,11 +1831,9 @@ fn parse_server_message(text: &str) -> ServerMessage {
         .map(str::to_string);
 
     // A request to keep the floor in this frame must reach the room before any
-    // generated reply sharing it.
-    if let Some(text) = message
-        .pointer("/serverContent/inputTranscription/text")
-        .and_then(Value::as_str)
-    {
+    // generated reply sharing it. In an interrupted frame it waits instead for
+    // the old turn to be torn down, below.
+    if !interrupted && let Some(text) = input_transcript {
         events.push(GeminiEvent::InputTranscript(text.to_string()));
     }
     if let Some(parts) = message
@@ -1878,19 +1883,21 @@ fn parse_server_message(text: &str) -> ServerMessage {
         events.push(GeminiEvent::UsageRecorded);
     }
 
+    // Gemini's interrupted turn ends after the interruption. A frame carrying
+    // both must preserve that order; candidate speech in the same frame owns
+    // the next reply, so it is recorded after the old turn is torn down.
+    if interrupted {
+        events.push(GeminiEvent::Interrupted);
+    }
     if message
         .pointer("/serverContent/turnComplete")
         .and_then(Value::as_bool)
         .unwrap_or(false)
     {
         events.push(GeminiEvent::TurnComplete);
     }
-    if message
-        .pointer("/serverContent/interrupted")
-        .and_then(Value::as_bool)
-        .unwrap_or(false)
-    {
-        events.push(GeminiEvent::Interrupted);
+    if interrupted && let Some(text) = input_transcript {
+        events.push(GeminiEvent::InputTranscript(text.to_string()));
     }
     if let Some(calls) = message
         .pointer("/toolCall/functionCalls")