diff --git a/quest/m1/audio-jitter-target/README.md b/quest/m1/audio-jitter-target/README.md index ecf0dde57e..40ee8c79d4 100644 --- a/quest/m1/audio-jitter-target/README.md +++ b/quest/m1/audio-jitter-target/README.md @@ -97,6 +97,7 @@ native playback against its budgets. - [Native A/V lag](/quest/m1/audio-jitter-target/native-av-lag.md) - `moq play --metrics` reports how far audio trails its presentation clock, so native rows grade `av_lag_ms` - [Native rendition delay](/quest/m1/audio-jitter-target/native-rendition-delay.md) - `moq play` holds each subscribed rendition's catalog `delay` plus its target, as the browser does - [Chromium Opus stalled share](/quest/m1/audio-jitter-target/opus-stalled-share.md) - the Chromium Opus browser rows' stalled-share ceiling is measured with the stretch, and passes +- [Native backpressure](/quest/m1/audio-jitter-target/native-backpressure.md) - native playback decodes audio only when the ring has room, holding the rest of the target encoded as the browser does ## Closes diff --git a/quest/m1/audio-jitter-target/grading.md b/quest/m1/audio-jitter-target/grading.md index a92db0201b..07c7f60a16 100644 --- a/quest/m1/audio-jitter-target/grading.md +++ b/quest/m1/audio-jitter-target/grading.md @@ -13,7 +13,9 @@ before the model lands, `test/audio-quality` grades: regression. - Dropped audio is the player's own count (`trimmed_ms`, `player_skips`). Speeding up is not a skip. -- No gate is fixed at zero underruns or zero stalls. +- No row's budget is fixed at zero underruns or zero stalls. Push-to-talk's + own zero gate, in `ptt-analyze.ts`, goes with + [Push-to-talk latency](/quest/m1/push-to-talk-latency.md), not here. - A congestion row, where the path stalls for longer than the target and then delivers the backlog, reports the audio dropped and how long audio takes to return to the clock. @@ -56,10 +58,11 @@ conformance corpus measures the target. Recommendations: -- Replay rows stay exact. Rewrite the note to say which direction is a - regression: a steady-state p50 lag that rises, or audio the player - trimmed. A p95 lag that rises through a stall, or an underrun count that - moves, is not one by itself. +- Replay rows stay exact, so any change to one still fails until it is + re-measured. Rewrite the note to say which re-measured moves are + regressions to fix rather than budget: a steady-state p50 lag that rises, + or audio the player trimmed. A p95 lag that rises through a stall, or an + underrun count that moves, is re-measured in. - Add the congestion case twice: a shaper profile that queues until packets drop, per the repository's impairment rule, and a replay trace where media trickles in late after a stall, which the audit of #8360 asked for. diff --git a/quest/m1/audio-jitter-target/native-av-lag.md b/quest/m1/audio-jitter-target/native-av-lag.md index 0f583476ee..3da047e0a0 100644 --- a/quest/m1/audio-jitter-target/native-av-lag.md +++ b/quest/m1/audio-jitter-target/native-av-lag.md @@ -12,9 +12,10 @@ playback decision does not change here. Decided 2026-11-09 (maintainer, /quest-complete on [moq-dev/moq#8361](https://github.com/moq-dev/moq/pull/8361)): ✅ keep the PR's metrics half and revert its playback changes (rejected: land its stretch -keyed to the lag at arrival). That stretch sped up into a thin buffer, taking -AAC `step` from 0 to 614 underruns a minute, and removed native's only -slow-down. It was built on the rule the +keyed to the lag at arrival). That stretch held audio on a clock that goes +stale after a path-delay step, taking AAC `step` from 0 to 614 underruns a +minute, where `main`'s slow-down had hidden the stale clock by settling +behind it. It was built on the rule the [epic](/quest/m1/audio-jitter-target/README.md) replaced the same day, so the PR's recorded decisions "accept AAC step underruns" and "p50 within one pitch period" go with it. diff --git a/quest/m1/audio-jitter-target/native-backpressure.md b/quest/m1/audio-jitter-target/native-backpressure.md new file mode 100644 index 0000000000..8856b759e6 --- /dev/null +++ b/quest/m1/audio-jitter-target/native-backpressure.md @@ -0,0 +1,63 @@ +# [M] Native playback decodes audio only when the ring has room + +## Goal + +Native playback (`moq_audio::decode::Consumer` feeding `moq play`'s sink) +decodes an audio frame when the browser's `Backpressure` would: while playing, +once the playhead is within the ring's floor of it, and, live, once the media +queued behind it covers the part of the target held encoded, so a backlog past +the target still reaches the ring to be caught up. The rest of the target +delay stays encoded upstream, so native memory follows the bytes actually +received and the ring never grows with the delay. Once this lands, Rust splits +the target between decoded PCM and encoded media the way JS does, and +`moq play` holds no decoded PCM outside the ring. + +## Plan + +Planned 2026-11-09 (maintainer, /quest-complete follow-up of +[moq-dev/moq#8362](https://github.com/moq-dev/moq/pull/8362): "quest-plan +backpressure like JS does. So we don't decode audio until there's space in +the ring buffer"). ✅ m1, in this epic (rejected: m2, m3). + +The browser already works this way: `Backpressure` in +`js/watch/src/audio/buffer.ts` decides when the decode loop may decode a frame, +holding the ring at its floor of PCM and the hold encoded in the container +consumer. Native decodes promptly instead: `Consumer::read` decodes each packet +as it is read, and the PCM waits in `play_audio`'s queue in +`rs/moq-cli/src/play/media.rs`. #8362 keeps a delay past the ring's budget in +sync by parking that PCM there until it is due. This quest replaces that queue +with the gate, so the hold stays encoded. + +Mirror the browser's rules, including the stalled refill that starts playback +with the whole target ahead of the playhead, rather than inventing new ones. +Share names and tests with the JS side where the two can match. + +The gate waits before decoding, never before reading. `Consumer::read` stamps +each packet's arrival for the jitter estimate (`target.observe` in +`rs/moq-audio/src/decode/consumer.rs`), and `Consumer::delay()` documents that +reads paced to the speaker make the estimate measure the pacing. Keep reading +and observing encoded packets as they arrive and hold them encoded, as +`js/watch` reads a frame and only then awaits `ring.wait()`. + +The gate has two inputs, and `play_audio` owns both, so the consumer needs +them passed in: + +- The playhead the JS worklet reports. Native can derive it from + `Sink::buffered()` (`rs/moq-audio/src/playback/sink.rs`) and the last + write's timestamp. +- The shared playout delay: the catalog `delay` plus the target, the largest + among the subscribed renditions, which the browser passes as + `sync.out.delay` and `play_audio` already holds the sink to through + `presentation.due`. `Consumer::delay()` is this track's jitter target alone; + a gate on it would decode at a depth the sink has already outrun and could + not replace #8362's hold. + +Keep the gate inside moq-audio's consumer if that stays additive; if it needs +a public API change, it moves to m2. + +Public API: additive if the gate stays inside `moq_audio::decode::Consumer`. +Wire: none. + +## Related + +- [Stretch model](/quest/m1/audio-jitter-target/stretch.md) - requires this gate, then changes the stalled refill mirrored here to a small cushion in both players diff --git a/quest/m1/audio-jitter-target/stretch.md b/quest/m1/audio-jitter-target/stretch.md index 05f10a3c15..c8395c11e1 100644 --- a/quest/m1/audio-jitter-target/stretch.md +++ b/quest/m1/audio-jitter-target/stretch.md @@ -52,9 +52,10 @@ Decisions for this quest: both are worth taking. - Order: ✅ after native backpressure (rejected: this first; one quest for both). The level this rule reads is everything received ahead of the - playhead, decoded or still encoded, and the backlog while audio trails - sits encoded upstream in the browser. One mirrored rule needs both players - to split the buffer the same way. + playhead, decoded or still encoded. The browser keeps the part of the + target above its ring's floor encoded, and a buffered player its whole + lookahead, while native decodes everything at once. One mirrored rule needs + both players to split the buffer the same way. - Clock step: ✅ deferred (rejected: plan it now; drop it). The shared clock's reference only moves earlier, so after a lasting path-delay step audio would settle behind a stale clock. Report what the `step` rows' lag @@ -72,10 +73,15 @@ What the audit of `main` found (2026-11-09, `10772522a5`): expands, fast-accelerates at 20 ms of excess, stays silent after an underrun until the whole target refills, parks on a target rise, and turns the stretch off in buffered mode. -- [moq-dev/moq#8361](https://github.com/moq-dev/moq/pull/8361) sped up into a - thin buffer and took AAC `step` from 0 to 614 underruns a minute. A - speed-up is admitted only while the buffer stays above the target by the - lag it removes. +- [moq-dev/moq#8361](https://github.com/moq-dev/moq/pull/8361) held native + audio on the clock and took AAC `step` from 0 to 614 underruns a minute. + Its own analysis: the clock's reference never moves later, so after the 5 + to 60 ms step the clock-aligned depth sits about 55 ms short of the + estimator's target, less than AAC's 160 ms PES spacing, and `main` had + hidden that by settling a burst behind. It already admitted a speed-up only + inside the excess, so that rule alone prevents nothing here. Under this + model the slow-down lets audio settle behind the stale clock rather than + underrun, which is the lag the deferred clock decision is about. Recommendations: @@ -117,18 +123,10 @@ decision now runs a filter. Public API: none expected. Wire: none. -## Waiting - -- moq-dev/moq: [moq-dev/moq#8380](https://github.com/moq-dev/moq/pull/8380) - merges, planning native backpressure - (`/quest/m1/audio-jitter-target/native-backpressure.md`). Then list that - quest under Required and delete this item. It asks native to mirror the - browser's stalled refill to the full target; this quest then changes the - refill in both. - ## Required - [The audio harness grades the stretch model](/quest/m1/audio-jitter-target/grading.md) - the gates that pushed toward zero underruns and zero lag are gone before this is measured against them +- [Native backpressure](/quest/m1/audio-jitter-target/native-backpressure.md) - native holds its ring at a floor and the rest of the target encoded, as the browser does, so one rule reads the same buffer in both ## Closes