Skip to content

test(transcode): own live source progress and clock - #7822

Merged
moq-bot[bot] merged 5 commits into
mainfrom
quest/m0/test-flakes-2/transcode-multi-rung
Nov 4, 2026
Merged

moq-bot[bot] merged 5 commits into
mainfrom
quest/m0/test-flakes-2/transcode-multi-rung

Conversation

@kixelated

@kixelated kixelated commented Nov 4, 2026 •

Copy link
Copy Markdown
Collaborator

Problem

The live multi-rung fixture publishes groups every 100 ms while native codec threads run independently. Under encoder load it overruns the shared 16-item feed and receives Cancel while draining the first output group.

A diagnostic CPU workload on each rung's first encode reproduced the exact line-550 panic in 3/3 runs. Both listeners reported Lagged(3) and cancelled group 0 after one frame. No encoder restart or source-decode failure occurred.

Approach

  • Reuse the source catalog fixture and drive the production ladder/rung handlers directly. Poll each demanded handler to attach both feed listeners before publishing media.
  • Install a manual clock and publish the next group only after every rung reads the current group through FIN. Advance time by that group's media duration afterward.
  • Share the fixture across software, hardware, and VAAPI live tests. Check all three group sequences, all five frames per group, timestamps, and Annex-B framing.
  • Delete the completed quest and its epic link.

Impact

  • Public API: none.
  • Wire: none.
  • Production behavior, feed capacity, and subscriber delay budgets are unchanged.

Alternatives

Paused tokio alone advances while native codec threads are busy. Wall-clock pacing depends on host load. Directly driving the existing handlers avoids introducing production readiness hooks; existing end-to-end tests still cover run.

Validation

  • With the same temporary encoder CPU workload: old fixture failed 3/3; replacement passed 10/10. The workload was 500,000,000 black_box(1u64) iterations once on the first encode request for workers with config.resize.is_some(). All diagnostic code was removed afterward.
  • quest gate just ci: 341 selected Rust tests passed, plus Rust formatting/docs/clippy and the other selected checks. The flake check alone failed building github-runner because unshare cannot write /proc/self/uid_map on this host.
  • Unchanged origin/main at 13f5e43743: nix develop --command quest gate nix flake check --print-build-logs failed identically, on the same /nix/store/k6j6qw86x77lz6dnl2wqbxi86cjy31yw-github-runner.drv.
  • Final clean-source load repeat: 20/20 passed. quest gate python3 .tmp/loaded-live.py runs just rs test -p moq-transcode --lib -E 'test(=tests::live_multi_rung)' --stress-count 20 --no-fail-fast on CPU 0 alongside 16 CPU workers pinned to that core. The helper joins every worker before exit.
  • quest check --base origin/main: 601 documents OK.
  • Hardware execution is unavailable on this host; hardware probes skip it.

Review

mechanical fix by quest land watch: merged main at 1b68a5983c5940792d3689e6078f3ccedca94bea with the quest merge driver (head c53c46a)

Claude Code (claude-opus-5-5, medium effort), quest review round 1: no findings on head 941d324735 against origin/main at 13f5e43743.

Head 1a16912613 merges origin/main at 3f39969175 to clear a conflict. The quest link-list merge driver resolved quest/m0/test-flakes-2/README.md. git diff origin/main...HEAD -- rs matches the reviewed head's Rust diff exactly. The only PR-diff change is surrounding context in the epic README, so the merge needs no new review. After the merge, just rs test -p moq-transcode --lib -E 'test(=tests::live_multi_rung)' --stress-count 5 passed 5/5.

Follow-ups

Restore developer-host Nix builder UID mappings before claiming a strict flake pass. The existing non-strict skip is already handled by moq-dev/moq#7763; no change to that policy is proposed here.

(Written by GPT-6.1 Sol; merge update written by Claude Opus 5.5)

kixelated and others added 3 commits November 3, 2026 21:49
Co-Authored-By: GPT-6.1 Sol <noreply@openai.com>
…2/transcode-multi-rung

Co-Authored-By: GPT-6.1 Sol <noreply@openai.com>
@kixelated kixelated changed the title moq-transcode's live multi-rung test survives load test(transcode): own live source progress and clock Nov 4, 2026
@kixelated
kixelated marked this pull request as ready for review November 4, 2026 06:10
@coderabbitai

coderabbitai Bot commented Nov 4, 2026 •

Copy link
Copy Markdown
Contributor

Explain Change →

No actionable comments were generated in the recent review.

Walkthrough

The live multi-rung tests now use a shared helper that builds a pipeline, publishes three five-frame groups, and checks each requested rung’s output. The helper uses a manual clock, and the software, hardware, and VA-API tests call it. The kio dev-dependency now enables the manual feature. The README entry and planning document for the flaky test were removed.

Severity of issue fixed: Low

Merge Risk: Minimal · up to c53c4

The changes are limited to test coordination and planning-document cleanup. No actionable merge-blocking issue was found; hardware execution remains unverified.

Pre-merge checks: Passed 5
Check name Status Explanation
Title check Passed The title clearly identifies the transcode test change: the fixture now controls live source progress and clock timing.
Description check Passed The description directly explains the flaky test problem, the fixture redesign, validation results, and scope of the changes.
Docstring Coverage Passed Docstring coverage is 100.00% which is sufficient. The required threshold is 80.00%. Docstring coverage is scoped to functions touched by this diff with cyclomatic complexity of at least 2. Documentat…
Linked Issues check Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check Passed Check skipped because no linked issues were found for this pull request.
Full details: Docstring Coverage

Explanation

Docstring coverage is 100.00% which is sufficient. The required threshold is 80.00%. Docstring coverage is scoped to functions touched by this diff with cyclomatic complexity of at least 2. Documentation-only edits and deletions within surviving functions are included; unchanged diff context and fully deleted functions are excluded. Analyzed 3 functions across 1 files.

Finishing Touches
Simplify code
  • Commit to this branch
  • Create a new PR
  • Autofix · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

Share
Recent review info
Run configuration
  • Source mode: Auto
  • Repository: Unavailable
  • Branch: Unavailable
  • File: No configuration file from this repository applied
  • Commit: Unavailable
  • Layers: Organization UI
  • Review profile: CHILL
  • Plan: Advanced
  • Run ID: 0c1d61af-005c-4538-bc4f-4996e7c57415
Commits

Reviewing files that changed from the base of the PR and between 1a16912 and c53c46a.

Files selected for processing: 1
  • quest/m0/test-flakes-2/README.md
Files skipped from review as they are similar to previous changes: 1
  • quest/m0/test-flakes-2/README.md

Included review availability: This review used your included allowance. Your plan provides up to 4 included reviews per hour; 3 remain after this review.

@kixelated kixelated left a comment

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Automated review by review (OpenAI)

Reviewed head 941d324.

No actionable issue found. The new fixture attaches each rung to the shared live feed before publishing, waits for every rung's group FIN before advancing the source, and checks all groups and frame timestamps. I also checked the manual-clock setup and the surrounding end-to-end coverage.

Verification: GitHub-only static review. I did not run the tests or verify hardware execution independently; the GitHub check was still running during review.

(Written by review (OpenAI))

@pullfrog pullfrog Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

✅ No new issues found.

Reviewed changes

Reviewed the full 4-file diff at head 941d32473: a test-only replacement of the flaky real-time live multi-rung fixture.

  • Deterministic fixture: live_groups installs a kio::clock::Manual clock and drives the production pipeline::Pipeline / rung::serve handlers group by group instead of pacing a real-time source. Each rung's serve future is polled once to attach it to the shared feed before any media is published.
  • Feed-safe sequencing: publishes the next source group only after every rung reads the current one through FIN, then advances the clock by that group's media duration. One group outstanding at a time keeps each rung's listener far under the 16-item feed capacity, so the Lagged/Cancel path the old fixture hit is structurally unreachable.
  • Stronger assertions: all three group sequences are checked per rung for sequence number, Annex-B start code, five-frame count, and exact source timestamps — the previous fixture only drained group 0.
  • Shared across backends: live_multi_rung, live_multi_rung_hardware, and vaapi_live_keeps_the_buffered_tail_in_its_group all route through the helper and now use start_paused = true; the VAAPI buffered-tail regression check is preserved (a tail moved into the next group would break the frame count/timestamp assertions).
  • Bookkeeping: removes the completed quest file and its epic link, and adds the dev-only manual feature to kio in rs/moq-transcode/Cargo.toml.

Pullfrog  | View workflow run | Using deepseek-v4.1-flash-1017 (free via Pullfrog for OSS) | 𝕏

@kixelated
kixelated added this pull request to the merge queue Nov 4, 2026
@github-merge-queue
github-merge-queue Bot removed this pull request from the merge queue due to a conflict with the base branch Nov 4, 2026
@moq-bot

moq-bot Bot commented Nov 4, 2026

Copy link
Copy Markdown
Contributor

quest land watch: the merge queue removed this pull request at 1a16912 because it conflicts with main (merge_conflict).

The land fixer works on it in the background, and the watcher re-enqueues it at the back of the queue once it is fixed or judged flaky, or hands it back here with what it needs.

(written by quest land watch)

@moq-bot
moq-bot Bot enabled auto-merge November 4, 2026 17:45
@moq-bot

moq-bot Bot commented Nov 4, 2026

Copy link
Copy Markdown
Contributor

quest land watch: armed auto-merge on this pull request at c53c46a, so it joins the back of the merge queue once its required checks pass, with main at 1b68a59 merged in by the quest merge driver.

(written by quest land watch)

@moq-bot
moq-bot Bot added this pull request to the merge queue Nov 4, 2026
Merged via the queue into main with commit 77d6abc Nov 4, 2026
9 of 10 checks passed
@moq-bot
moq-bot Bot deleted the quest/m0/test-flakes-2/transcode-multi-rung branch November 4, 2026 19:03
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant