Repository navigation
feat(mux)!: moq_mux::sync keeps tracks decoupled on one wall-clock reference - #8272
Conversation
Quest-Claim: 979859ff485444d7
…ference (WIP) Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…ayhead per broadcast Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
|
Warning Review limit reachedYou've used all free OSS reviews for now. Wait for the free limit to reset to keep reviewing this public repository. Next included review available in 4 minutes. View limit details
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. |
kixelated
left a comment
There was a problem hiding this comment.
Automated review by review (OpenAI)
Reviewed a0c65b7. No actionable findings. Checked per-track earliest references, reset/release and target resolution, held-replacement offsets, three-kind transition mapping, and CLI audio/video integration. Audio now steers its buffer toward the shared due time without owning video's anchor; speaker playhead tracking remains separate for metrics. This matches the explicit direction of steady-state sync with independent progress during stalls.
Verification: GitHub-only static review against the previous timeline implementation and JS sync semantics, including regression tests and updated benchmark wiring. No builds, tests, audio-device playback, timing reproduction, or benchmarks run.
There was a problem hiding this comment.
✅ No new issues found.
Reviewed changes
moq_mux::syncreplacesmoq_mux::timeline— per-track arrival anchors with the earliest as the reference,leftretained for already-queued media,reset/drop releasing a track's part,set_targetdriving the resolved delay, andcontainer::Kind(Audio/Video/Data) replacing the module's ownKind.- Generalized transition machinery — declared ends and encoder restarts now span three roles; paired restarts take their origin from the earliest
sourceplan, andSegment.heldshifts arrivals from media that waited behind an old tail. moq playaudio no longer owns the clock — each write folds the frame's arrival, sets the track's jitter target, and steers the sink towarddue(end) - now(clamped to the device floor and age budget); thePlayheadmoved intoplay/timeline.rsand is reset per broadcast, with--metricsreporting the resolved clock delay.- Wake handling and tests — audio task completion now wakes video when the released target lowers the delay; new mocked-time tests cover stalled audio, reset handoff, target selection, held replacements, and a data-track paired restart.
- Renames and docs — bench
timeline→syncacrossCargo.toml,rs/justfile, and nightly;doc/bin/cli.md,doc/lib/rs/moq-mux.md, crate docs, and quest references updated;quest/m2/mux-sync.mddeleted.MIGRATE.mdis correctly left for the release cut.
I traced the anchor/left fallback, reset/release paths, transition pairing and pruning, the held shift, Playhead locking and per-generation isolation, and the audio stretch-target math; no stale moq_mux::timeline callers remain in the tree. Local cargo test --release -p moq-mux --lib sync:: was blocked by a held target-directory lock in this sandbox, so CI is the build/test authority for this head.
deepseek-v4.1-flash-1017 (free via Pullfrog for OSS) | 𝕏
|
quest land watch: the merge queue removed this pull request at a0c65b7 because its The land fixer works on it in the background, and the watcher re-enqueues it at the back of the queue once it is fixed or judged flaky, or hands it back here with what it needs.
…ent_consumes_only_the_request
PASS [ 0.012s] ( 2156/13788) moq-ffi server::transport_tests::converts_supported_transports
PASS [ 0.092s] ( 2157/13788) moq-ffi session::tests::maps_native_auth_connect_errors
PASS [ 0.092s] ( 2158/13788) moq-ffi session::tests::maps_closed_errors_without_flattening_the_reason
PASS [ 0.363s] ( 2159/13788) moq-ffi producer::metadata_tests::dropping_a_group_binding_ends_its_unfinished_group
PASS [ 0.689s] ( 2160/13788) moq-ffi media::timebase_tests::an_unused_timebase_does_not_withhold_the_catalog
PASS [ 0.262s] ( 2161/13788) moq-ffi socket::tests::protocols_are_independent_and_endpoint_close_wakes_both_queues
PASS [ 0.561s] ( 2162/13788) moq-ffi origin::supervision_tests::readers_derived_from_a_routed_broadcast_report_the_origin_panic
PASS [ 0.895s] ( 2163/13788) moq-ffi media::read_tests::encoded_media_preserves_breaks
PASS [ 0.275s] ( 2164/13788) moq-ffi socket::tests::cancelled_accept_wait_leaves_the_queue_and_rejection_is_one_shot
PASS [ 0.274s] ( 2165/13788) moq-ffi socket::tests::cancelling_a_dial_resets_its_incoming_but_the_endpoint_can_dial_again
PASS [ 0.197s] ( 2166/13788) moq-ffi test::a_closed_route_leaves_its_replacement_alone
PASS [ 0.285s] ( 2167/13788) moq-ffi socket::tests::tcp_halves_outlive_the_stream_and_preserve_fin
PASS [ 1.002s] ( 2168/13788) moq-ffi media::catalog_tests::catalog_handle_updates_sections
PASS [ 0.199s] ( 2169/13788) moq-ffi socket::tests::track_subscription_survives_stream_release_and_has_no_default_age_limit
PASS [ 0.376s] ( 2170/13788) moq-ffi socket::tests::dropping_an_unfinished_tcp_writer_resets_with_a_deliberate_cause
PASS [ 0.286s] ( 2171/13788) moq-ffi socket::tests::tcp_reset_is_not_fin_and_does_not_close_another_session
PASS [ 0.104s] ( 2172/13788) moq-ffi test::a_hidden_route_is_invisible_until_announced
PASS [ 0.189s] ( 2173/13788) moq-ffi test::an_instance_pins_request_and_follow
PASS [ 0.292s] ( 2174/13788) moq-ffi test::a_new_route_at_a_path_restarts_its_followers
PASS [ 0.190s] ( 2175/13788) moq-ffi test::an_exact_route_yields_no_requests_and_a_prefix_route_no_broadcast
PASS [ 0.191s] ( 2176/13788) moq-ffi test::abort_after_finish_reaches_group_consumer
PASS [ 0.191s] ( 2177/13788) moq-ffi test::accepting_a_request_after_close_is_refused
PASS [ 0.192s] ( 2178/13788) moq-ffi test::abort_after_finish_keeps_the_track_handle
PASS [ 0.190s] ( 2179/13788) moq-ffi test::announce_and_withdraw_toggle_discovery
PASS [ 0.293s] ( 2180/13788) moq-ffi test::a_waiting_ready_ends_on_abort
FAIL [ 1.193s] ( 2181/13788) moq-ffi media::read_tests::raw_media_catalog_warmup_reaches_the_subscription
stdout ───
running 1 test
test media::read_tests::raw_media_catalog_warmup_reaches_the_subscription ... FAILED
failures:
failures:
media::read_tests::raw_media_catalog_warmup_reaches_the_subscription
test result: FAILED. 0 passed; 1 failed; 0 ignored; 0 measured; 321 filtered out; finished in 1.18s
stderr ───
thread 'media::read_tests::raw_media_catalog_warmup_reaches_the_subscription' (1988497) panicked at rs/moq-ffi/src/media.rs:1635:13:
assertion `left == right` failed: video=false
left: [4400000, 5000000]
right: [4400000, 4700000, 5000000]
note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace
Cancelling due to test failure: 23 tests still running
PASS [ 0.302s] ( 2182/13788) moq-ffi test::a_track_is_adopted_by_one_writer
PASS [ 0.306s] ( 2183/13788) moq-ffi test::a_failed_finish_ends_the_track_with_its_cause
PASS [ 0.017s] ( 2184/13788) moq-ffi test::audio_rejects_bad_init_bytes
PASS [ 0.019s] ( 2185/13788) moq-ffi test::announced_hides_dot_paths_unless_asked
PASS [ 12.659s] ( 2186/13788) moq-cli::integration report::a_late_sink_reads_the_final_report
PASS [ 3.069s] ( 2187/13788) moq-ffi audio::tests::retired_audio_producers_keep_clean_tails_for_late_readers
PASS [ 0.109s] ( 2188/13788) moq-ffi test::announced_filters_patterns_and_reports_captures
PASS [ 0.108s] ( 2189/13788) moq-ffi test::audio_raw_publish_rejects_bad_frames
PASS [ 0.111s] ( 2190/13788) moq-ffi test::ascending_subscription_reads_a_cached_backlog_oldest_first
PASS [ 0.110s] ( 2191/13788) moq-ffi test::audio_consumer_follows_a_sibling_broadcast_reference
PASS [ 0.110s] ( 2192/13788) moq-ffi test::audio_subscribe_configuration_boundary
PASS [ 0.301s] ( 2193/13788) moq-ffi test::an_unpaced_writer_evicts_a_slow_reader
PASS [ 0.120s] ( 2194/13788) moq-ffi test::audio_raw_publish_from_many_threads
PASS [ 0.777s] ( 2195/13788) moq-ffi session::tests::new_rejects_invalid_config
PASS [ 1.291s] ( 2196/13788) moq-ffi media::timebase_tests::containers_of_one_source_keep_their_relative_media_time
PASS [ 1.185s] ( 2197/13788) moq-ffi session::tests::setting_tokens_refuses_an_empty_token
PASS [ 1.191s] ( 2198/13788) moq-ffi session::tests::new_accepts_the_defaults_and_known_versions
PASS [ 13.727s] ( 2199/13788) moq-cli::integration rtsp::every_rtsp_session_is_a_fresh_broadcast
PASS [ 14.636s] ( 2200/13788) moq-cli::integration listen::an_export_stage_ending_during_drain_waits_for_every_session
PASS [ 2.879s] ( 2201/13788) moq-ffi server::lifecycle_tests::accept_on_a_foreign_thread
PASS [ 17.332s] ( 2202/13788) moq-cli::integration import::a_short_import_drains_its_media_to_a_waiting_subscriber
PASS [ 17.703s] ( 2203/13788) moq-cli::integration import::import_delivers_the_catalog_finish_at_eof
PASS [ 17.701s] ( 2204/13788) moq-cli::integration report::import_reports_count_what_each_rendition_wrote
────────────
Summary [ 28.268s] 2204/13788 tests run: 2203 passed, 1 failed, 151 skipped
FAIL [ 1.193s] ( 2181/13788) moq-ffi media::read_tests::raw_media_catalog_warmup_reaches_the_subscription
stdout ───
running 1 test
test media::read_tests::raw_media_catalog_warmup_reaches_the_subscription ... FAILED
failures:
failures:
media::read_tests::raw_media_catalog_warmup_reaches_the_subscription
test result: FAILED. 0 passed; 1 failed; 0 ignored; 0 measured; 321 filtered out; finished in 1.18s
stderr ───
thread 'media::read_tests::raw_media_catalog_warmup_reaches_the_subscription' (1988497) panicked at rs/moq-ffi/src/media.rs:1635:13:
assertion `left == right` failed: video=false
left: [4400000, 5000000]
right: [4400000, 4700000, 5000000]
note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace
warning: 11584/13788 tests were not run due to test failure (run with --no-fail-fast to run all tests, or run with --max-fail)
error: test run failed
mbx[cache]: object cache: 44 hits, 179 misses, 90 bypassed; 0 B downloaded, 0 B uploaded, 6.2 GiB stored locally
mbx[cache]: 6m 12s session; 1m 57s estimated compiler time avoided (summed across compilations)
mbx[cache]: bypass reasons: 89 unsupported-crate-type, 1 unportable-native-link
mbx[cache]: Cargo artifact reuse and CI cache archive transfers are not included above
mbx[savings]: has quietly disposed of 13.7 TiB logical of build leftovers. nobody saw anything.
error: recipe `test` failed on line 367 with exit code 100
error: recipe `test-changed` failed on line 86 with exit code 100
##[endgroup]
##[error]ci: test rs failed
ci: seconds per module
192 check rs
372 test rs (failed)
5 check biome
191 check js
17 test js
10 check markdown
13 check gh
323 test gh
46 check flake
1 check drill
5 check quest
1 check ts_paths
27 check shell
0 check large_files
0 check text
1 check toml
0 check nix
0 check justfile
564 wall time
error: recipe `ci` failed on line 119 with exit code 100
##[error]Process completed with exit code 100.
##[group]Run actions/upload-artifact@cf430e030ddbb5b0abf93d22962f4752f3646cd9
with:
name: cargo-timings
path: target/cargo-timings/cargo-timing-*.html
if-no-files-found: ignore
retention-days: 14
compression-level: 6
overwrite: false
include-hidden-files: false
archive: true
env:
BASE: 79960a9dd64e48a2fe51a06fc1a3e9b4692dfb22
CHECK_FAIL_FAST: 1
NEXTEST_TEST_THREADS: 24
MBX_CACHE_LINKS: 1
MBX_REMOTE_URL:
##[endgroup]
With the provided path, there will be 4 files uploaded
Artifact name is valid!
Root directory input is valid!
Uploading artifact: cargo-timings.zip
Beginning upload of artifact content to blob storage
Uploaded bytes 219686
Finished uploading artifact content to blob storage!
SHA256 digest of uploaded artifact is b7ac63dd3f4323c840a5e56bc91dc5d4d620ffc380c7b45cec9393539d9fd904
Finalizing artifact upload
Artifact cargo-timings successfully finalized. Artifact ID 13626696125
Artifact cargo-timings has been successfully uploaded! Final size is 219686 bytes. Artifact ID is 13626696125
Artifact download URL: https://github.com/moq-dev/moq/actions/runs/42502335110/artifacts/13626696125
Post job cleanup.
##[start-action display=Run jdx/mr-boxington-action@88acf2bb50552c7c508cbac13e5f8c576b95b1de;id=__02ee65a1-2453-4945-9f1d-3fda0167bbcd.mbx]
##[end-action id=__02ee65a1-2453-4945-9f1d-3fda0167bbcd.mbx;outcome=skipped;conclusion=skipped;duration_ms=0]
Post job cleanup.
[command]/nix/store/lg46w7hrjx7kylsh6645c9l975i8h43b-git-2.55.0/bin/git version
git version 2.55.0
Temporarily overriding HOME='/var/cache/github-runner-work/_temp/13e73b94-5cad-4daf-b809-abc6517733ab' before making global git config changes
Adding repository directory to the temporary git global config as a safe directory
[command]/nix/store/lg46w7hrjx7kylsh6645c9l975i8h43b-git-2.55.0/bin/git config --global --add safe.directory /var/cache/github-runner-work/moq/moq
Removing SSH command configuration
[command]/nix/store/lg46w7hrjx7kylsh6645c9l975i8h43b-git-2.55.0/bin/git config --local --name-only --get-regexp core\.sshCommand
[command]/nix/store/lg46w7hrjx7kylsh6645c9l975i8h43b-git-2.55.0/bin/git submodule foreach --recursive sh -c "git config --local --name-only --get-regexp 'core\.sshCommand' && git config --local --unset-all 'core.sshCommand' || :"
Removing HTTP extra header
[command]/nix/store/lg46w7hrjx7kylsh6645c9l975i8h43b-git-2.55.0/bin/git config --local --name-only --get-regexp http\.https\:\/\/github\.com\/\.extraheader
[command]/nix/store/lg46w7hrjx7kylsh6645c9l975i8h43b-git-2.55.0/bin/git submodule foreach --recursive sh -c "git config --local --name-only --get-regexp 'http\.https\:\/\/github\.com\/\.extraheader' && git config --local --unset-all 'http.https://github.com/.extraheader' || :"
Removing includeIf entries pointing to credentials config files
[command]/nix/store/lg46w7hrjx7kylsh6645c9l975i8h43b-git-2.55.0/bin/git config --local --name-only --get-regexp ^includeIf\.gitdir:
[command]/nix/store/lg46w7hrjx7kylsh6645c9l975i8h43b-git-2.55.0/bin/git submodule foreach --recursive git config --local --show-origin --name-only --get-regexp remote.origin.url
A job completed hook has been configured by the self-hosted runner administrator
##[group]Run '/nix/store/gx2v6q84b0w43bafm0whkdx93qyxbpps-ci-cache-gc.sh/bin/ci-cache-gc.sh'
shell: /nix/store/1mv3qz005gkbalxfghk4y3s8ayv4d2dl-bash-interactive-5.3p15/bin/bash --noprofile --norc -e -o pipefail {0}
##[endgroup]
evicted 0 objects and 0 action results (0 B logical); 50.0 GiB logical remain
removed 0 learned incremental units and 2 checkout directories (3.7 KiB logical); 1 units (220.9 MiB logical) remain
evicted 0 objects and 0 action results (0 B logical); 50.0 GiB logical remain
removed 3 target directories (34.4 GiB logical, 2 abandoned and 1 live); 0 B logical remain
Cleaning up orphan processes
(written by quest land watch) |
|
codex, gpt-6.1-sol, medium effort land fixer note:
(written by codex, gpt-6.1-sol, medium effort) |
|
codex, gpt-6.1-sol, medium effort land fixer: FAIL [ 1.193s] ( 2181/13788) moq-ffi media::read_tests::raw_media_catalog_warmup_reaches_the_subscription
stdout ───
running 1 test
test media::read_tests::raw_media_catalog_warmup_reaches_the_subscription ... FAILED
failures:
failures:
media::read_tests::raw_media_catalog_warmup_reaches_the_subscription
test result: FAILED. 0 passed; 1 failed; 0 ignored; 0 measured; 321 filtered out; finished in 1.18s
stderr ───
thread 'media::read_tests::raw_media_catalog_warmup_reaches_the_subscription' (1988497) panicked at rs/moq-ffi/src/media.rs:1635:13:
assertion `left == right` failed: video=false
left: [4400000, 5000000]
right: [4400000, 4700000, 5000000]
note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace
(written by codex, gpt-6.1-sol, medium effort) |
# Conflicts: # quest/m2/api-shape/media.md

Completes quest/m2/mux-sync.md and deletes it.
Problem
moq_mux::timeline, the native playout clock behindmoq play, was audio-master. While audio played, only the speaker's position moved the anchor. A stalled speaker that came back on a fresh sink re-pinned the anchor and held video back with it. That contradicts the 2026-11-05 ruling on moq-dev/moq#7941: lip sync in the steady state, desync under congestion, and one slow track never stalls or moves another. The module also carried its own copy ofKind.Approach
This ports
js/watch/src/sync.ts(per-track timelines on one reference) tomoq_mux::sync:Track::set_target). While no track holds one, it uses theProducer::newdelay. Audio holds its jitter target. Video has no estimator, so it holds none, which keeps today's latency (100 ms video-only, audio's estimate otherwise).moq playfolds each audio frame's arrival into the clock. Each write sets the sink's stretch target todue(end) - now, clamped to the 50 ms device floor and the age budget. In the steady state that is the jitter target. After a stall it is shallower than what the sink holds, so the callback speeds up, or past NetEq's fast threshold trims, until audio is back on the clock. Video never waits for it.plan, applied atapply), so media we held back doesn't read as early and drag the other tracks ahead.Kind. The declared-end and encoder-restart machinery now coverscontainer::Kind(audio, video, data). A paired restart takes its common origin from the member plan with the earliest source rather than always audio's. The two agree in every existing case.moq playasPlayheadand feeds--metrics.--metrics'delayis now the clock's resolved delay, matching the browser'ssync.out.delay.Tests use mocked time throughout. New ones:
sync::stalled_audio_no_longer_holds_video: a late audio burst and a reset leave video's schedule alone.play::media::stalled_audio_no_longer_holds_video: end to end throughplay_audioandVideo::run. Audio stalls, its backlog lands late and forces a fresh sink, and every picture still presents atstart + delay + pts.sync::a_video_burst_moves_audio_onto_the_live_edge,a_reset_hands_the_schedule_to_the_tracks_left,the_delay_is_the_largest_target_held,media_held_behind_a_tail_leaves_the_schedule_alone,a_data_track_joins_a_paired_restart.The audio-owns-the-anchor tests are gone. The video burst tests with a speaker now assert that the burst reaches live past a trailing speaker. The playhead tests move to
play/timeline.rs.Impact
moq-mux:moq_mux::timelineis nowmoq_mux::sync. Release had an unrelatedmoq_mux::timeline(the join-interval one), so the cut's MIGRATE entry should say both are gone. Migration:timeline::Kind→moq_mux::container::Kind.Producer::tracktakes it, andKind::Datais accepted.Producer::video(ts, now)→track.received(ts, now)on the videoTrack.Producer::audio(Audio { .. }, now)→track.received(ts, arrival)on the audioTrack, plustrack.set_target(jitter_target). Steer the speaker towardproducer.due(end)yourself.Producer::video_restartedandProducer::audio_restarted→track.reset().Producer::audio_stopped→ drop orretirethe track.timeline::Audiois removed.Consumer::target()→Consumer::delay(), which always returns the resolved delay.Consumer::playhead()is removed: the speaker tracks what it is sounding.Producer::new(delay)now means the delay while no track holds a target.moq play: no flag changes. A stalled audio track no longer holds video.--metricswritesdelayon every sample. Audio-only steady-state depth after a write is one packet deeper than before, because a frame's end now sounds when the clock says it is due.timelineis renamed tosync(sync_present,sync_received), inrs/justfileand the nightly list.Alternatives
--delay's auto default (100 ms) as a target. This would raise A/V latency above audio's own estimate, so video holds no target.Choices made without asking
syncto audio and video. The quest asks forcontainer::Kind, which includesData.moq play's two playback halves keep a private two-variantKind, since the player has no data half.delayfield is how a publisher states that offset (seedoc/concept/audio-jitter.md).Follow-ups
moq play's use of the catalog renditiondelayagainstdoc/concept/audio-jitter.md'smax(delay + target). The browser holds it, and this PR doesn't change the native side. Recommendation: plan it, since a slow video encoder otherwise plays behind audio natively.Models
Review
Rewritten by
quest reviewafter each round.01a1ba42-5819-7ae2-8aae-77b707195bca, round 3The previously reviewed implementation is unchanged, and static review of the merge resolution found no new issues; no builds or tests were run.🤖 Generated with Claude Code
(Written by Claude Opus 5.5 with high effort)