Problem
When the candidate listens to the interviewer on speakers, the microphone re-captures that audio. The interviewer's words then arrive on both channels, and nothing in the app notices:
transcript.service.ts:92 attributes speaker purely by channel name (ch_0 -> Other, everything else -> Self). There is no diarization and no cross-channel comparison.
mergeAdjacentTranscripts (transcript.service.ts:52-73) only merges same-speaker runs, so the echo becomes a second, independent block labelled with the candidate's name.
The visible symptom is duplicated and misattributed transcript lines. The damaging one is quieter: an echoed question lands as a recent Self final, so skipDueToRecentSelf (transcript.service.ts:203-204) suppresses the live suggestion for the very question that was just asked, and an echoed mic partial trips blockedByPartial for up to SELF_PARTIAL_STALE_MS (15 s, consts.ts:33).
Both suppressions are silent. Speaker users do not get a degraded product - they get one that goes quiet exactly when it matters.
And the docs tell users to use speakers
There is no headphones requirement anywhere in these repos. grep -ri "headphone|earphone|headset" across client/src, client/README.md, client/SPEC.md, client/CLAUDE.md and hero/src/content/docs/ returns one hit, and it is about a headset failing mid-interview.
The published docs say the opposite:
hero/src/content/docs/best-practices.md:53 - the app "reads whatever audio is playing through your system speakers automatically"
best-practices.md:60 - "Play a short audio clip through your speakers and verify the loopback channel picks it up", as a pre-interview check
usage.md:28, troubleshooting.md:98 - same framing
So this is not a requirement to relax. It is a defect users are being walked into, undocumented.
Why a real fix is cheap here
The app already captures the exact reference signal an echo canceller needs: ch_0 is a loopback of the render endpoint, literally what the speakers are playing. And nobody ever listens to the mic stream; it only feeds an ASR. So the goal is not to cancel the echo cleanly, it is to detect it and stop sending it.
Plan
All of it on one branch, feat/echo-suppression (#113), in the order the work has to happen.
Constraints the design has to respect
Four, three from reviewing the plan against the source and one measured by the probe.
-
The lag can be negative. The acoustic path is always mic-after-speaker, but what the gate sees is arrival order at the worklet, and Chromium's getDisplayMedia loopback path carries its own latency. A one-sided 0..MAX search would find no peak on exactly the setups this feature exists for. A signed window in turn means the mic has to be delayed to keep decisions causal - a decision rule of the form Em[f] < ERL * Er[f-d] with negative d reads a reference frame that has not arrived yet.
-
Peak correlation cannot tell coupling from noise. Measured with the probe: 0.53 on pure silence and 0.57 on two independent bursty signals, because the search takes the max over ~120 candidate lags and the max of many correlations is biased upward. A CORR_MIN of 0.5 calls both of those coupled - and a false "coupled" on a headphone user is what leads the gate to cut a microphone that was never echoing anything. Peak prominence (peak minus median lag) separates them: 0.28 unrelated against 0.87-1.13 for a real echo.
-
The gate must start OPEN and close only once coupling is proven. The opposite rule - close whenever the reference is active until estimates converge - applies to every user for the first few seconds, including the headphone users the product works for today, and would cut their microphone during the first interviewer utterance. Never regress the working configuration to speed up the broken one.
-
Layer 2 must not delete a genuine repeat-back. A candidate repeating the question back ("So you're asking about my Kafka work?") is routine, and comes after the interviewer. Similarity alone would silently delete it, so a time condition is mandatory - and it has to be interval overlap rather than start-alignment, because both timestamps are first-partial arrivals on two independent ASR sessions whose interim latency differs by hundreds of milliseconds.
Two smaller ones, both found in review and both easy to get wrong:
- Dropping an echoed
Self entry in ingest() must also clear selfPartialTranscript. That field is cleared only on the Self-final path (:110-113), so a bare return strands a sub-threshold opening fragment ("So tell", 2 tokens, below any minimum-token floor) which then re-arms blockedByPartial for 15 s - reintroducing the exact suppression this issue is about.
echo-gate.ts should pace its cross-correlation on a frame counter, not performance.now(). Wall-clock pacing is fine for a live capture but makes the class untestable from synthetic input, and the gate needs unit tests.
Not a cost saving
_periodic_credit_deduction (backend/app/services/asr_service.py:592-618) deducts on a wall-clock timer, halved for the dual connection. Credits are a function of session length, not of what is in the audio. Gating is a correctness fix only.
Problem
When the candidate listens to the interviewer on speakers, the microphone re-captures that audio. The interviewer's words then arrive on both channels, and nothing in the app notices:
transcript.service.ts:92attributes speaker purely by channel name (ch_0->Other, everything else ->Self). There is no diarization and no cross-channel comparison.mergeAdjacentTranscripts(transcript.service.ts:52-73) only merges same-speaker runs, so the echo becomes a second, independent block labelled with the candidate's name.The visible symptom is duplicated and misattributed transcript lines. The damaging one is quieter: an echoed question lands as a recent
Selffinal, soskipDueToRecentSelf(transcript.service.ts:203-204) suppresses the live suggestion for the very question that was just asked, and an echoed mic partial tripsblockedByPartialfor up toSELF_PARTIAL_STALE_MS(15 s,consts.ts:33).Both suppressions are silent. Speaker users do not get a degraded product - they get one that goes quiet exactly when it matters.
And the docs tell users to use speakers
There is no headphones requirement anywhere in these repos.
grep -ri "headphone|earphone|headset"acrossclient/src,client/README.md,client/SPEC.md,client/CLAUDE.mdandhero/src/content/docs/returns one hit, and it is about a headset failing mid-interview.The published docs say the opposite:
hero/src/content/docs/best-practices.md:53- the app "reads whatever audio is playing through your system speakers automatically"best-practices.md:60- "Play a short audio clip through your speakers and verify the loopback channel picks it up", as a pre-interview checkusage.md:28,troubleshooting.md:98- same framingSo this is not a requirement to relax. It is a defect users are being walked into, undocumented.
Why a real fix is cheap here
The app already captures the exact reference signal an echo canceller needs:
ch_0is a loopback of the render endpoint, literally what the speakers are playing. And nobody ever listens to the mic stream; it only feeds an ASR. So the goal is not to cancel the echo cleanly, it is to detect it and stop sending it.Plan
All of it on one branch,
feat/echo-suppression(#113), in the order the work has to happen.echoCancellationandautoGainControlon 2-3 machines; recorddelayMs(with sign),correlation,prominenceanderlDb. Layer 1 is tuned against whatever these settle on, so measuring afterwards means tuning twice. Needs a person and real speakers; cannot run in CI.AudioContext, a two-input worklet, and a delay-aligned energy gate onch_1(audio-capture-graph.ts+ a dependency-freeecho-gate.ts).ingest(), gated on time overlap (echo-dedup.ts).herodocs and adocumentation-dialog.tsxentry on speaker vs headphone capture; fix the stale "deduplicates overlapping segments" claim atCLAUDE.md:63.Constraints the design has to respect
Four, three from reviewing the plan against the source and one measured by the probe.
The lag can be negative. The acoustic path is always mic-after-speaker, but what the gate sees is arrival order at the worklet, and Chromium's
getDisplayMedialoopback path carries its own latency. A one-sided0..MAXsearch would find no peak on exactly the setups this feature exists for. A signed window in turn means the mic has to be delayed to keep decisions causal - a decision rule of the formEm[f] < ERL * Er[f-d]with negativedreads a reference frame that has not arrived yet.Peak correlation cannot tell coupling from noise. Measured with the probe: 0.53 on pure silence and 0.57 on two independent bursty signals, because the search takes the max over ~120 candidate lags and the max of many correlations is biased upward. A
CORR_MINof 0.5 calls both of those coupled - and a false "coupled" on a headphone user is what leads the gate to cut a microphone that was never echoing anything. Peak prominence (peak minus median lag) separates them: 0.28 unrelated against 0.87-1.13 for a real echo.The gate must start OPEN and close only once coupling is proven. The opposite rule - close whenever the reference is active until estimates converge - applies to every user for the first few seconds, including the headphone users the product works for today, and would cut their microphone during the first interviewer utterance. Never regress the working configuration to speed up the broken one.
Layer 2 must not delete a genuine repeat-back. A candidate repeating the question back ("So you're asking about my Kafka work?") is routine, and comes after the interviewer. Similarity alone would silently delete it, so a time condition is mandatory - and it has to be interval overlap rather than start-alignment, because both timestamps are first-partial arrivals on two independent ASR sessions whose interim latency differs by hundreds of milliseconds.
Two smaller ones, both found in review and both easy to get wrong:
Selfentry iningest()must also clearselfPartialTranscript. That field is cleared only on the Self-final path (:110-113), so a barereturnstrands a sub-threshold opening fragment ("So tell", 2 tokens, below any minimum-token floor) which then re-armsblockedByPartialfor 15 s - reintroducing the exact suppression this issue is about.echo-gate.tsshould pace its cross-correlation on a frame counter, notperformance.now(). Wall-clock pacing is fine for a live capture but makes the class untestable from synthetic input, and the gate needs unit tests.Not a cost saving
_periodic_credit_deduction(backend/app/services/asr_service.py:592-618) deducts on a wall-clock timer, halved for the dual connection. Credits are a function of session length, not of what is in the audio. Gating is a correctness fix only.