[kixelated]

Problem

After fixing Firefox’s audio-backend launch in moq-dev/moq#8072, the lifecycle suite passed once in 147 seconds, then failed replacement playback with only 58/69 samples satisfying the audio-tone assertion. The repository requires immediately disabling flaky tests with an investigation follow-up.

Approach

Quarantine only the Firefox replacement case from the default selection. Chromium and WebKit keep it. Explicit --media-cases replacement runs retain every content, sync, switch, identity, and fallback assertion, so investigation remains possible.

Impact

• Test selection only; no public API, wire, production behavior, assertion, or tolerance changes.
• Default Firefox runs print an explicit skip with the audio-work link.

Alternatives

Retries and looser tone thresholds would hide the missing samples. Disabling every browser’s replacement case would drop coverage beyond the observed failure.

Follow-ups

Trace the missing Firefox replacement-tone samples alongside moq-dev/moq#8058, determine whether playout or the measurement loses them, and restore default selection when the cause is fixed. The audio PR is related work, not a proven fix for this failure.

Validation

• just test media --media-engines firefox: passed twice with lifecycle, all six negative controls, and both native-publisher cases. Final lifecycle run took 102 seconds. AAC publish/codec change was capability-skipped because Chromium has no platform AAC encoder.
• The final run emitted a separate replacement SKIP cell and console line beside the lifecycle PASS, keeping the quarantine visible in the report.
• just ci: passed at the final head.

Review

Claude Code / Claude Opus 5.5, configured medium effort, round 2 reviewed 1093113d0f: clean. Fixed round 1's finding that a passing lifecycle cell hid the quarantined case by recording it as a separate explicit SKIP. Replacement assertions and explicit investigation runs remain intact.

(Written by GPT-6.1 Sol with xhigh effort)

[coderabbitai]

Warning

Review limit reached

You've used all free OSS reviews for now. Wait for the free limit to reset to keep reviewing this public repository.

Next included review available in 26 minutes.

Check out review usage here.

View limit details

Limit details: You’ve used all 4 included reviews currently available.

Learn how review limits work.

Review configuration:

Run configuration

• Source mode: Auto
• Repository: Unavailable
• Branch: Unavailable
• File: No configuration file from this repository applied
• Commit: Unavailable
• Layers: Organization UI
• Review profile: CHILL
• Plan: Advanced
• Run ID: e6157253-28f9-429a-a18f-c587dcc279e2

Commits

Reviewing files that changed from the base of the PR and between f2f9bc9 and 1093113.

Files selected for processing: 3

• test/interop/README.md
• test/interop/clients/js/media.ts
• test/interop/interop.sh

• Autofix · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

Share

• X
• Mastodon
• Reddit
• LinkedIn

[kixelated]

Automated review by review (OpenAI)

Reviewed commit: 1093113
Base: f2f9bc9

No actionable findings in this diff. The direction is sound: quarantine the observed Firefox replacement flake without loosening the assertions.

• test/interop/clients/js/media.ts:133-169: only the default fixture run with a Firefox player drops replacement; explicit --cases replacement, Chromium/WebKit defaults, and native-publisher selection remain intact.
• test/interop/interop.sh:2205-2209: the quarantine marker becomes a distinct skip record rather than counting the omitted case as a pass. The normal run result and negative controls are preserved.
• The replacement content, sync, instance-identity, switch-bound, and fallback assertions remain unchanged, and the README retains an explicit investigation command and follow-up link.

Verification limits: GitHub-only static review of all three changed files, the case-dispatch paths, replacement assertions, and result-recording helper. I did not run quest context, builds, browser tests, or reproduce the audio flake. The Check workflow was still in progress when checked. This review does not establish that the underlying Firefox audio issue is fixed.

[pullfrog]

ℹ️ No critical issues — the quarantine is correct and scoped as described. The only gap is the follow-up tracking below.

Reviewed changes

• Firefox-only case filter — the default (no --cases) selection drops replacement when the player engine is Firefox, leaving Chromium, WebKit, explicit --media-cases, negative controls, and native runs untouched.
• Quarantine marker — media.ts prints a tab-separated QUARANTINE line on stderr naming replacement and linking #8058.
• Skip reporting — run_media parses that marker from a passing run's log and records an extra skip case plus an explicit SKIP line, so the quarantine shows in the interop report instead of vanishing.
• Docs — the media README documents the quarantine, the explicit --media-cases replacement rerun, and that its content, sync, switch, and fallback assertions still run there.

ℹ️ The quarantine has no tracked follow-up

The only durable pointer is #8058, which the PR body itself calls related work rather than a fix, so nothing schedules the investigation once that PR merges. This repo records every quarantined test as a quest/m0/test-flakes-2 child with a restore condition (see reload-close-drain.md and the Quarantined pending quest/... comment in js/net/src/connection/reload.test.ts), and the epic ledger notes each new quarantine as it lands.

Technical details

# Track the Firefox replacement quarantine

## Affected sites
- `test/interop/clients/js/media.ts:164-168` — the marker links only #8058, not a work item with a restore condition.
- `test/interop/README.md:531-535` — says to investigate alongside the audio-playout work, then restore, but names no tracker.
- `quest/m0/test-flakes-2/README.md` — the epic ledger has no entry for this quarantine, though it already carries media/interop flakes (`pulse-dbus.md`, `ietf22-fallback.md`).

## Required outcome
- A child quest (or issue) names the failing assertion (the `audio tone` agreement in Firefox's `replacement <version>` window), the reproducing command (`just test media --media-engines firefox --media-cases replacement`), and the restore condition: remove the filter at `media.ts:136` and the marker at `media.ts:164` once it passes repeatedly.

## Suggested approach (optional)
- Add `quest/m0/test-flakes-2/<name>.md`, a Required bullet in that epic's README, and point the README paragraph and the `QUARANTINE` detail at it (or at an issue) rather than at #8058 alone.

  ^{｜ Fix it ➔ ｜ View workflow run ｜ Using deepseek-v4.1-flash-1017 (free via Pullfrog for OSS) ｜ 𝕏}