Echo gating, step 0: explicit mic constraints and a coupling probe - #112
Closed
alpha5611331 wants to merge 3 commits into
Closed
Echo gating, step 0: explicit mic constraints and a coupling probe#112alpha5611331 wants to merge 3 commits into
alpha5611331 wants to merge 3 commits into
Conversation
Both getUserMedia call sites passed only a deviceId, so echoCancellation, noiseSuppression and autoGainControl ran on whatever Chromium currently defaults to. All three default to true today, so this changes no behaviour - it stops the behaviour changing on its own under a Chromium version bump, and gives the echo work one place to flip them from. AGC is split out as a named constant because it is the flag most likely to move: it raises gain through quiet passages, which amplifies re-captured interviewer audio on a speaker setup. The no-device case is an object with no deviceId rather than `audio: true`, which would have dropped the flags along with it. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Measures how much of the interviewer's audio the microphone re-captures when the candidate is on speakers. Reports three numbers per machine: the signed arrival-order delay between the two channels, the correlation peak at that lag, and the echo return loss. The sign matters and is the reason the search window is two-sided. The acoustic path is always mic-after-speaker, but what is measured here is arrival order at the worklet, and Chromium's getDisplayMedia loopback path carries its own latency - so on a machine where it is the slower of the two, the reference arrives after the echo it explains. The window searched is wider than any gate would ship with, so a peak sitting at the edge is distinguishable from a window that is too small; the summary warns when that happens. An estimate is only accepted while the reference is actually active. A silent run reached a correlation of 0.53 - two noise floors correlate - so a peak height alone cannot tell coupling from silence. Manual, like taskbar-probe.mjs: it needs a desktop session, real speakers and someone to play audio into them, so it stays out of test/run.mjs. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
|
|
Verifying the correlator against synthetic signals turned up the more important defect. Delay recovery is exact, including the sign: +120, +300, -150, -250 and 0 ms all come back to the frame, and the ERL matches the injected gain. But the coupling verdict was wrong in the dangerous direction. The search takes the max over ~120 candidate lags, and the max of many correlations is biased upward, so unrelated signals score far higher than intuition suggests: 0.53 on pure silence, 0.57 on two independent bursty signals. A CORR_MIN of 0.5 calls both of those coupled, and a false "coupled" on a headphone user is what would lead a gate to cut a microphone that was never echoing anything. Prominence - the peak's height above the median lag - separates them cleanly: 0.28 for the unrelated pair against 0.87-1.13 for a real echo. Both ends of that gap are optimistic, so the threshold is a starting point to be re-derived from real runs, and the raw numbers are printed every second regardless. Also: reject unknown arguments and a non-positive --seconds. A mistyped --noaec was silently ignored, which runs with echo cancellation ON and reports a plausible number for the configuration you were trying to rule out; a non-numeric --seconds reached setTimeout as NaN and ended the run before it started, which reads like a headphone result. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
4 tasks
Member
Author
|
Closed automatically by GitHub when the head branch was renamed from |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Refs #111. First step of the echo-gating work: the constraint change everything else is tuned against, and the instrument that does the tuning. No gating yet - nothing in this PR changes what reaches the ASR.
1. State the microphone processing constraints explicitly
Both
getUserMediacall sites inlive-transcription.service.tspassed only adeviceId, soechoCancellation,noiseSuppressionandautoGainControlran on Chromium's implicit defaults.This changes no behaviour today - all three already default to
true. The point is that they stop moving on their own under a Chromium version bump, and that there is one place to flip them from once the probe says which way they should go. AGC is a named constant because it is the flag most likely to move: it raises gain through quiet passages, which amplifies re-captured interviewer audio on a speaker setup.The no-device case is an object with no
deviceIdrather thanaudio: true, which would have dropped the flags along with it.2. A manual probe for the coupling
test/manual/echo-probe.mjsdrives a real Electron session and reports three numbers per machine:delayMscorrelationerlDbTwo things it is built to catch:
The sign of the delay. The acoustic path is always mic-after-speaker, but what is measured is arrival order at the worklet, and Chromium's
getDisplayMedialoopback path carries its own latency. On a machine where it is the slower of the two, the reference arrives after the echo it explains. The search window is two-sided, and deliberately wider (-400..+800 ms) than any gate would ship with - so a peak sitting at the edge is distinguishable from a window that is too small. The summary warns when that happens, and again when the delay comes back negative.Silence masquerading as coupling. An estimate is only accepted while the reference is actually active. A silent run of an earlier version of this probe reached a correlation of 0.53 with nothing playing at all - two noise floors correlate - which is above the 0.5 that looked like a reasonable
CORR_MIN. A peak height alone cannot tell coupling from silence, and the gate will have to carry the same reference-active condition.It also drives the A/B this PR's first half exists to enable:
Manual, like
taskbar-probe.mjs: it needs a desktop session, real speakers and someone to play audio into them, so it stays out oftest/run.mjs.Verification
pnpm lint,tsc -b,pnpm test:mainall green.aec=true ns=true agc=true), runs the correlator, and prints a summary. With nothing playing it correctly reports no correlated frames and points at theref%column.Reviewer note
Worth a look at the acceptance criterion in
renderer.js(estimate()) and at the two-sided search window - those are the two places where getting it wrong produces a plausible-looking number rather than an obvious failure.