Skip to content

fix(mux,hang): a backlog join starts within the max delay in Rust and JS - #7518

Merged
moq-bot[bot] merged 10 commits into
mainfrom
quest/m1/cut-audit/catalog-parity
Nov 2, 2026
Merged

moq-bot[bot] merged 10 commits into
mainfrom
quest/m1/cut-audit/catalog-parity

Conversation

@kixelated

@kixelated kixelated commented Nov 1, 2026 •

Copy link
Copy Markdown
Collaborator

Summary

A player joining a backlog now starts at the same group in Rust (moq-cli, moqsrc) and the browser, and that group is within the max delay of the live edge. This finishes quest/m1/cut-audit/catalog-parity.md and deletes it.

The rule is the same in both consumers. Until the first item is read, the start moves past each group whose first frame trails the newest content by more than the max delay plus the warmup. It never drops the group covering the live edge less the min delay and the warmup, which is moq-dev/moq#7311's start promise. Once reading begins, nothing changes.

The quest's scenario (five one-second groups of ten frames, a one-frame sixth group at 5 s, all landed before the first read, 1.5 s budget) now starts at the 4.0 s group in both. On main, Rust played the whole backlog from 0, and JS started at the 3.0 s group. #7311's warming rule had moved JS from its old first-frame start to the reach.

Decisions

Maintainer answers, 2026-11-01:

  1. Which start a backlog join takes.
  2. Ordering with moq-dev/moq#7311.

Changes

  • Rust (rs/moq-mux/src/container/consumer.rs): Consumer::poll_event_once gets a start skip, start_skip, that runs until the first item. The blocking-group skip and its lookahead loops move into skip_to, poll_next_group, and poll_newest, so the start skip shares them.
    • The start skip looks only at the cursor's immediate successor, and skips nothing until that group's first frame lands, as JS does (#groups[1].start === undefined). A successor whose frames are still in flight may start within the budget, so jumping on to the next group with data could drop it.
    • The start skip walks the whole backlog in one call and reads the newest content once. skip_to drops only the prefix below its target (discard_front), so a long backlog's start is linear rather than quadratic.
  • JS (js/hang/src/container/consumer.ts): before the first item, #checkMaxDelay's warming exception keeps only the group covering the start's target (#startTarget(#reach(delay))). After the first item, feat(mux)!: a Delay { min, max } sets where a consumer starts and how late it falls #7311's rule stays: a group lives until its successor starts a whole budget behind.
  • Paired tests:
    • The quest's scenario: Rust a_buffered_backlog_starts_within_the_budget and JS a buffered backlog starts within the budget, each with the sixth group open and finished.
    • A successor without frames yet: Rust a_backlog_start_waits_on_a_successor_without_frames and JS a backlog start waits on a successor without frames. The fifth group (4 s) arrives before its frames, so the start keeps the fourth and plays on into the fifth. Without the fix, Rust drops the whole fifth group and starts at 5 s.
    • The warmup protection: Rust the_start_keeps_a_group_that_began_before_the_reach_but_covers_it, the counterpart of the existing JS the start keeps a group that began before the reach but covers it. With the guard disabled, the Rust test fails.
  • Adjusted tests:
    • Eight moq-mux tests wrote a backlog before their first read but were testing a skip after the start. They now read first, so they still test that skip. For startup_skips_groups_without_data, the newest group sits exactly one budget past group 7's first frame, so the start stays there.
    • a_long_group_blocked_behind_a_newer_one_is_not_cut_short reads before the newer group lands. Its doc no longer says a join's backlog behaves that way.
    • moq-video the_tail_drains_from_both_decoders now uses a budget longer than its 5 s backlog. Its comment already said the budget keeps every cached group, and that stopped being true at 4 s.
  • Docs: the Rust Consumer and Delay::max docs, JS Delay.max, both MIGRATE.md entries for feat(mux)!: a Delay { min, max } sets where a consumer starts and how late it falls #7311, and doc/lib/rs/hang.md all said the max delay "never moves the start". They now describe the backlog start. There's also a moq-mux CHANGELOG entry under Fixed.

Choices made without asking

  • The protection target moves with the edge. Before the first read, both consumers compute the covering target as the newest content less min(min, max) + warmup, rather than the value fixed when feat(mux)!: a Delay { min, max } sets where a consumer starts and how late it falls #7311's start wait ended. This also covers a floored start (the subscription names a group), such as moq-video's max-delay lane with Start::Latest. That lane has no fixed target, and a_max_delay_decode_error_fails_the_read needs its group 0 kept.
  • JS's warming rule after the first read stays as feat(mux)!: a Delay { min, max } sets where a consumer starts and how late it falls #7311 left it. Applying the covering-target rule there too would protect a stalled reader's group with no time bound.
  • The quest is deleted, not rewritten. The brief asked for the Plan to be updated "either way". The guide says the PR that finishes a quest deletes it, and this PR finishes it, so the decisions are recorded here and in the commit.

Checks

On 103e448a37, after the review fixes:

  • cargo nextest run -p moq-mux -p moq-video -p moq-audio: 2294 passed.
  • cargo clippy -p moq-mux --all-targets -D warnings and cargo fmt --check.
  • In js/hang: bun test src/container/ (173 pass), tsc --noEmit, and biome check on the changed test.
  • quest check --base origin/main: ok.

Earlier, on 7f681fa:

  • cargo test -p moq-mux --lib: 1954 passed.
  • cargo test -p moq-video -p moq-audio: passed.
  • bun test src/container/ in js/hang: 172 passed.
  • quest gate just check origin/main on 7f681fa808: passed (rs check and test, 4385 tests; js check and test; markdown, quest, shell).
  • quest check --base origin/main: ok.

Review

Earlier: quest review --base origin/main, Codex gpt-6.1-sol at medium effort, on head 7f681fa808. 0 findings.

Online reviewer (Pullfrog) on 7f681fa, one thread, fixed in e9ca4d0:

  • consumer.rs:819: the start skip targeted the first later group with a frame, so a successor whose first frame was still in flight was skipped and discarded, unlike JS, which waits on its successor's start. Confirmed: on the old code, the new paired test drops the whole fifth group and starts at 5 s. Fixed as described under Changes.

The fix's local review, in a new session (quest review --base origin/main, Codex gpt-6.1-sol at medium effort):

  • Round 1, head e9ca4d0: one finding, fixed. Each skip discarded through retain_mut over every pending group, so a long backlog's start was quadratic in one poll. Fixed in 5f06574 with discard_front, which drains only the prefix.
  • Round 2, head 5f06574: one finding, fixed. Each skip re-read poll_newest, which rescans trailing groups without frames. Fixed in 103e448: the start skip walks the backlog in one call and reads the newest content once.
  • Round 3, head 103e448a37: 0 findings.

The head adds a conflict-free merge of origin/main that touches quests alone.

Follow-ups

  • Steady-state parity. After the start, JS's age skip still drops buffered groups whose first frame trails by more than the budget. Rust skips only a blocking group, measured by reach. This PR aligns only the join.
  • A successor that never gets a frame. While the cursor's successor has no first frame, both consumers keep the cursor's group, and reading it ends the start rule, so the join can start behind the budget. Holding the start until the successor's frame lands would need a bound for a group that never gets one.
  • The steady-state skip's per-skip scans. After the start, the blocking-group skip still calls poll_newest once per skipped group, as on main.
  • Hole measure parity. JS proves a missing group stale by head.latest - presentedEnd > maxDelay. Rust uses the newest timestamp minus the next group's start (aged).

(Written by Claude Opus 5.5)

🤖 Generated with Claude Code

kixelated and others added 3 commits November 1, 2026 01:18
A prototype for the maintainer's decision, reverted in the next commit.

Until the first item, the Rust container consumer's start moves past every
group whose first frame trails the newest content by more than the max delay,
the measure @moq/hang uses, so the quest's backlog starts at the 4 s group in
both. Once reading begins, only a blocking group is skipped, as before. Paired
tests in both consumers, and eight moq-mux tests that wrote a backlog before
the first read now read first.

It breaks two moq-video dual-decoder tests on main
(the_tail_drains_from_both_decoders, a_max_delay_decode_error_fails_the_read):
the warmup decoder's earlier group is fresh by its reach but its first frame
trails the newest by more than max + warmup, so the start drops the group the
warmup needs. #7311 adds the container warmup that would protect it.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…he maintainer

Reverts the prototype in the previous commit, which breaks moq-video's
dual-decoder warmup on main, and rewrites the quest with what the attempt
found: the wire judges a group by its reach, JS by its first frame, and Rust
skips only a blocking group. It lists the two start measures with their test
fallout and recommends the browser's measure at the start, after
#7311 lands the container warmup the Rust start needs.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@kixelated kixelated changed the title Rust and JS skip a backlog by the same measure quest: record the backlog-start options for Rust and JS consumers Nov 1, 2026
kixelated and others added 2 commits November 1, 2026 14:34
The maintainer picked option A on 2026-11-01: a backlog join takes JS's
first-frame measure at the start, so it begins within the budget, with
#7311's warmup protection.

Until the first item, both container consumers move the start past each
group whose first frame trails the newest content by more than the max
delay plus the warmup, unless that group covers the live edge less the
min delay and the warmup. The quest's scenario (five one-second groups
and a sixth at 5 s, 1.5 s budget) now starts at the 4 s group in both;
#7311 had moved JS to the 3 s group and Rust still played from 0.

- Rust: a start skip in Consumer::poll_event_once, with the skip and
  lookahead loops factored into skip_to, poll_next_group, poll_newest.
- JS: #checkMaxDelay keeps only the group covering the start's target
  before the first item; after it, #7311's warming rule is unchanged.
- Paired tests: a_buffered_backlog_starts_within_the_budget and
  the_start_keeps_a_group_that_began_before_the_reach_but_covers_it in
  Rust, matching consumer.skips.test.ts and consumer.warmup.test.ts.
- Eight moq-mux tests that wrote a backlog before the first read now
  read first, and moq-video's tail-drain test uses a budget past its
  backlog.
- Delete the finished quest quest/m1/cut-audit/catalog-parity.md.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@kixelated kixelated changed the title quest: record the backlog-start options for Rust and JS consumers fix(mux,hang): a backlog join starts within the max delay in Rust and JS Nov 1, 2026
@kixelated
kixelated marked this pull request as ready for review November 1, 2026 22:53
@coderabbitai

coderabbitai Bot commented Nov 1, 2026 •

Copy link
Copy Markdown
Contributor

Warning

Review limit reached

You've used all free OSS reviews for now. Wait for the free limit to reset to keep reviewing this public repository.

Next included review available in 42 minutes.

Check out review usage here.

View limit details

Limit details: You’ve used all 4 included reviews currently available.

Learn how review limits work.

Review configuration:

Run configuration
  • Source mode: Auto
  • Repository: Unavailable
  • Branch: Unavailable
  • File: No configuration file from this repository applied
  • Commit: Unavailable
  • Layers: Organization UI
  • Review profile: CHILL
  • Plan: Advanced
  • Run ID: dbba0f12-5592-43c1-b14a-f80da95de586
Commits

Reviewing files that changed from the base of the PR and between 88e6990 and 10badca.

Files selected for processing: 10
  • doc/lib/rs/hang.md
  • js/hang/MIGRATE.md
  • js/hang/src/container/consumer.skips.test.ts
  • js/hang/src/container/consumer.ts
  • quest/m1/cut-audit/README.md
  • quest/m1/cut-audit/catalog-parity.md
  • rs/moq-mux/CHANGELOG.md
  • rs/moq-mux/MIGRATE.md
  • rs/moq-mux/src/container/consumer.rs
  • rs/moq-video/src/decode/consumer.rs
  • Autofix · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

Share

@pullfrog pullfrog Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Important

Two things to settle before merge: the new start skip changes what Start::Oldest returns for a cached backlog (recorders/exporters) while their docs still promise the whole cache, and Rust's start_skip diverges from JS when the next group has arrived but carries no frame yet.

Reviewed changes

  • Rust Consumer::start_skip (rs/moq-mux/src/container/consumer.rs): before the first item, the cursor moves past each group whose first frame trails the newest content by more than max_delay + warmup, keeping the group covering newest - (min(min_delay, max_delay) + warmup). The blocking-group skip moved into skip_to / poll_next_group / poll_newest.
  • JS #checkMaxDelay (js/hang/src/container/consumer.ts): before the first item, the warming exception keeps a group only when its successor starts past #startTarget(#reach(delay)); the post-first-item #7311 rule is unchanged.
  • Tests: paired Rust/JS backlog test, a Rust warmup-coverage test, eight moq-mux tests read before the newer groups land, and moq-video the_tail_drains_from_both_decoders raises its budget from WARMUP to 2 * WARMUP.
  • Docs: the Consumer/Delay docs, both MIGRATE.md entries, doc/lib/rs/hang.md, and the moq-mux CHANGELOG describe the backlog start; the quest is deleted.

⚠️ Start::Oldest no longer decodes everything cached, but its docs still say it does

The rule is applied to every unfloored container consumer, including moq-video/moq-audio Start::Oldest (the default), which sets no range and reaches the mux with a bounded Delay::max. A recording, export, or anything reading a complete track that once read the whole cache now starts within max_delay of the newest content, silently dropping the head of the backlog. The moq-video tail-drain test had to raise its budget to keep its "every cached group" assertion, which is the tell. Either an explicit Start::Oldest should bypass the start skip, or the contract docs need to change.

Technical details
# `Start::Oldest` + bounded `Delay::max` skips the head of a whole-buffered backlog

## Affected sites (stale statements, none updated by this PR)
- `rs/moq-video/src/decode/consumer.rs:59` "The oldest group the track still holds, decoding everything cached."
- `rs/moq-video/src/decode/consumer.rs:62-63` "the default because dropping media a caller has not asked to drop is the worse mistake."
- `rs/moq-video/src/decode/consumer.rs:104` "Defaults: ... every cached group ..."
- `rs/moq-video/src/decode/consumer.rs:623-627` test doc "The default reads everything the track holds." (test now sets `max: Some(10s)`)
- `rs/moq-audio/src/decode/consumer.rs:52` "Decode every cached group."
- `rs/moq-audio/src/decode/consumer.rs:85-86` "[`Delay::max`] ... before the subscription skips it; it adds no buffering."
- `rs/moq-ffi/src/media.rs:34` and `:50-51`
- `rs/moq-c/src/api.rs:710`, `rs/moq-c/src/decode.rs:31-32`, `rs/moq-c/src/video.rs:183`, `rs/moq-c/src/audio.rs:128`
- `rs/moq-mux/src/container/consumer.rs:3562-3563` (same file as the change): "the max delay bounds lateness, not the start."

## Required outcome
- Decide whether an explicit `Start::Oldest` (a caller asking for the whole cache) should opt out of the start skip. If yes, gate `start_skip` on the unfloored/`Start::Latest` path only. If no, the docs above must say the backlog start drops groups past `Delay::max`.
- Fix the stale statements in the same PR (AGENTS.md: outdated docs are fixed inline).

## Suggested approach
`options.start == Start::Oldest` reaches `Reader::open` (`rs/moq-video/src/decode/consumer.rs:307-357`) as an unfloored subscription. The floored `Start::Latest` path is already noted as covered in the PR description; `Start::Oldest` is not floored, so it is currently unavoidable from the caller side.

Pullfrog  | Fix all ➔ | Fix 👍s ➔ | View workflow run | Using deepseek-v4.1-flash (free via Pullfrog for OSS) | 𝕏

Comment thread rs/moq-mux/src/container/consumer.rs Resolved
kixelated and others added 4 commits November 1, 2026 15:39
The start skip targeted the first later group with a frame, so a group
that had arrived with its first frame still in flight was skipped and
discarded, even when that frame would land within the budget. It now
looks only at the cursor's immediate successor and skips nothing until
that group's start is known, as @moq/hang's consumer waits on its
successor's start. Paired tests in Rust and JS cover the case; the Rust
one drops the whole fifth group without the fix.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
skip_to discarded through `retain_mut`, which visits every pending group,
so a start that skips a long backlog one group at a time was quadratic in
one poll. `pending` is in sequence order, so the groups below the target
are its prefix: `discard_front` drains just those, with the same fault
check `discard` makes.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
`start_skip` now walks the whole backlog in one call and reads the newest
content once, rather than once per skipped group. Trailing groups whose
frames are still in flight no longer make each skip rescan them. The
newest content lies in the highest group with a timestamp, which a start
skip never drops, so one read serves every skip.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

@pullfrog pullfrog Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

✅ No new issues found.

Reviewed changes

Reviewed the commits since the prior pullfrog review (7f681fa): the wait-on-successor fix and two start-skip perf refactors, all in rs/moq-mux.

  • Rust start_skip waits on the successor's first frame (e9ca4d0): it now breaks when the cursor's immediate successor has no buffered frame yet, rather than skipping across it and discarding it, matching JS's #groups[1].start === undefined bail. This is the fix for the prior inline thread.
  • skip_to extracted, poll_newest read once (103e448): the start skip walks the whole backlog in one call and caches the newest content, so a long backlog is not rescanned per skip.
  • discard_front / discard_fault / discarded extracted (5f06574): skip_to drains only the prefix below its target via VecDeque::drain(..count), making a long backlog's start linear instead of quadratic, while discard keeps its predicate-driven behavior for the other call sites.
  • Paired tests added: Rust a_backlog_start_waits_on_a_successor_without_frames with JS a backlog start waits on a successor without frames, plus adjusted moq-mux startup/skip tests and a merge of origin/main.

The refactor preserves discard's fault detection (discard_fault on the lowest dropped group) and the start rule's reach guard; I traced the pending invariant that keeps the cached poll_newest group alive across skips, and that discard_front's prefix equals the previous sequence < target predicate on a sequence-ordered pending.

Pullfrog  | View workflow run | Using deepseek-v4.1-flash (free via Pullfrog for OSS) | 𝕏

@kixelated
kixelated enabled auto-merge (squash) November 2, 2026 00:12
@moq-bot
moq-bot Bot merged commit 7c2d1dd into main Nov 2, 2026
1 check passed
@moq-bot
moq-bot Bot deleted the quest/m1/cut-audit/catalog-parity branch November 2, 2026 01:48
kixelated added a commit that referenced this pull request Nov 3, 2026
#7518's backlog-start check recomputed the live edge over every buffered
group on each skip, making a bulk skip before the first read quadratic
again. The loop already holds that edge as the span's max, so pass it in.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant