Repository navigation
feat(fuzz): fuzz the media parsers untrusted publishers reach - #7313
Conversation
Quest-Claim: ee02af7b1680a382
Add moq-mux-fuzz (fmp4, mkv, ts, flv, cmaf, annexb, catalog) and moq-rtmp-fuzz (amf0). Each media target bounds heap traffic linearly in the input through a per-thread counting allocator; amf0 asserts the borrowed receive views allocate nothing. The bodies live in hidden `fuzz` modules so each crate's tests replay seeds and committed regressions on stable, and nightly.yml runs every target. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
# Conflicts: # sh/rs/select.test.py
|
Minimized reproduction inputs for the findings in the PR body, base64. Write one to a file and replay it with
|
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
WalkthroughAdds seven media fuzz targets to Merge Risk:
|
Codex Review SummaryThis comment shows the latest Codex review activity on this pull request.
ℹ️ About Codex in GitHubYour team has set up Codex to review pull requests in this repo. Reviews are triggered when you
Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings. |
kixelated
left a comment
There was a problem hiding this comment.
Automated review by review (OpenAI)
Reviewed commit: c2bfb3f5e6289905db1f4c9fb69a515d29d6b29d.
No new actionable bugs found in the changed code. Sharing target bodies between libFuzzer and stable regression tests is a sound direction; the thread-local allocation accounting and existing target-discovery workflow avoid unnecessary parallel harnesses.
The already documented parser failures and allocation amplification remain follow-up work, not new findings from this review (see the reproduction inputs). The acknowledged red nightly targets will stop at their first failure until those fixes land.
Verification: static inspection of target bodies, allocation accounting, seeds/replay, feature/workspace wiring and nightly integration. I did not run tests, builds, fuzzing or the nightly ASan path.
* fix: reject sgpd entry counts past the box A version 0 entry of an unknown grouping type reads to the end of the box, so every later entry decodes empty for free, and a 20-byte sgpd declaring 0xffff0000 entries grew a Vec past 3 GB. Every entry takes at least a byte, so a count above the remaining bytes is now OutOfBounds. Found by moq's media fuzz targets (moq-dev/moq#7313). Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * fix: keep the sgpd entry capacity cap Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
iloc rejects zero-size extents after an item's first; webm-iterable and the SPS parsers are fixed in moq-mux and reported upstream; mp4-atom#264 merged, so the bump waits on a 0.17.x release containing it. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
|
Landing
(Written by Claude Opus 5.5) |
|
quest land removed this pull request from the queue: Conflicts with the base: .github/workflows/nightly.yml |
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
kixelated
left a comment
There was a problem hiding this comment.
Automated review by review (OpenAI)
Reviewed commit: c3eccc46032566cdf24bef9b3efe92f2e475b417, compared with my prior review at c2bfb3f5e6289905db1f4c9fb69a515d29d6b29d, accounting for the main merges.
No new actionable bugs found. The meaningful addition is rs/moq-mux/fuzz/regressions/mkv/short-block-header: replaying the formerly crashing input through the existing stable regression harness is the right direction, without a second test framework. The inherited MKV block-header fix bounds the header reads; the shared fuzz bodies and allocation accounting are unchanged. Workspace exclusions and the nightly matrix preserve the media targets alongside main's additions.
The earlier documented MKV panic is now covered by the new regression; the other documented parser/allocation issues remain tracked follow-up work, not new findings. The prior CodeRabbit documentation finding is addressed by the removal of its owning audit file (see discussion_r4431284114).
Verification: static comparison of PR-relative patches, regression discovery and MKV header handling. I did not run tests, builds or fuzzing. The GitHub connector refused the non-UTF-8 regression blob, so its exact bytes and runtime result were not independently verified.
main's #7468 added a parse-amplification measurement to quest/m2/media-fuzz.md, which this branch deletes. The measurement found three fMP4 cases that need fixing, so the quest now keeps that remaining work with the numbers instead of being deleted, and its m2 and import-metadata-policy links stay. - .config/hakari.toml: exclude moq-mux-fuzz and moq-rtmp-fuzz from the workspace-hack (#6985), like moq-net-fuzz and moq-quic-fuzz. - sps-overflow.md: H.264 now parses with h264-reader (#7398). - mp4-atom-bump.md: its prose wait moves into a ## Waiting section. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
kixelated
left a comment
There was a problem hiding this comment.
Automated review by review (OpenAI)
Reviewed commit: 8234e5b, compared with c3eccc4 accounting for main merges.
No new actionable bugs found. Adding the media fuzz crates to Hakari's traversal exclusions matches existing harness isolation. Retaining the fMP4 metadata quest with explicit memory and time goals is sensible: bounded input alone does not bound parsed state or planning cost. The harness and MKV regression remain unchanged; prior parser/allocation issues remain follow-up work.
Verification: static comparison of PR-relative patches, Hakari configuration, manifests and follow-up design. Tests, builds, fuzzing and allocation measurements were not independently run.
The maintainer picked landing #7313 as-is on 2026-11-03, with the parse amplification fixes in an m1 quest, because 116 bytes cost 584 MB on any fMP4 or HLS ingest. This branch now finishes quest/m2/media-fuzz.md and adds quest/m1/fmp4-metadata-limits.md for the three fixes. `quest mv` refused the move while #7313 claims the old path, so it was done with git mv and the links rewritten by hand. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
kixelated
left a comment
There was a problem hiding this comment.
Automated review by review (OpenAI)
Reviewed commit: c1b0999
No new actionable bugs found. Promoting the measured fMP4 amplification fixes to m1 and letting that quest set bounds independently of the later documentation policy is a coherent response to the reported exposure. The fuzz targets and previously documented failures are unchanged.
Static GitHub comparison only; I did not run tests, fuzzing or allocation measurements.
When the offset, length and index sizes are all 0, an iloc extent reads no bytes, so a 28-byte box decoded two items of 65,280 extents each and allocated about 6 MB. Only an item's first extent may now be zero-size; a second is OutOfBounds. A lone zero-size extent, naming the whole data reference, still decodes, even at the end of the box. Found by moq's media fuzz targets (moq-dev/moq#7313). Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Fuzz targets for the media parsers an untrusted publisher reaches, each asserting bounded heap work per input byte, wired into the nightly fuzz job, with committed regressions replayed in the merge gate. It does the fuzz-target work of quest/m2/media-fuzz.md. It also takes the parse-amplification measurement main added to that quest on 2026-10-31. The fixes that measurement calls for move to a new quest, quest/m1/fmp4-metadata-limits.md (see Base merge (2026-11-02) and Decision 7), and
media-fuzz.mdis deleted.Changes
rs/moq-mux/fuzz(moq-mux-fuzz):fmp4,mkv,ts,flv(container importers, fed in socket-sized writes chosen by a leading byte),annexb(H.264/H.265 Annex-B and AV1 OBU splitters throughTrackStream),cmaf(subscriber-sidefmp4::Wire::from_init+ fragment decode),catalog(hang and MSF catalog consumers read after each frame, then each rendition's container built).rs/moq-rtmp/fuzz(moq-rtmp-fuzz):amf0, which reads a message throughMessagePayload::amf0(), walks every value, and asserts zero allocations.moq_mux::fuzz::runcounts all heap traffic (cumulative bytes and allocation count, not just live memory) on the target's thread and fails an input above4 MiB + 1024 B/byteor4096 + 8 allocations/byte. Because it counts every allocation, it catches rebuffering or reparsing as well as holding memory. The constants come from what was measured: about 1.7 MB fixed setup (including the first compressed catalog and HLS groups), and about 4.6 KB per group opened by a 14-byte keyframe.fuzzmodules behind a newfuzzfeature, the same pattern as moq-net and moq-quic. Seeds come from the crates' existing fixtures.fuzz::tests::{seeds_are_accepted, regressions}replay them on stable undercfg(test), so they run in the merge gate.src/allocation.rsand gained cumulative counters. The import release tests and the fuzz targets now share it. moq-rtmp's existing AMF0 allocator is reused the same way.nightly.ymlfuzz matrix gained the 8 targets. Both crates were added to the workspace and to every--exclude *-fuzzlist (rs/justfile,sh/rs/select.sh,sh/rs/coverage.sh, and their tests).Public API: none (hidden
fuzzmodules behind an off-by-default feature). Wire: none.Findings from local fuzz runs
These come from short local runs (about 3 minutes per target, coverage-guided, stable toolchain, no ASan). Minimized inputs are in a PR comment. None are fixed here.
sgpddecodeilocdecodemeta/iref.rs:37blocks/block.rs:136,141rs/moq-mux/src/container/ts/step.rs:431(self.scan - self.at)scan < atsps.rs:186,197,204sps/mod.rs:303,sps/scaling_list.rs; scuffle-expgolomb 0.1.5amf0(60M runs) andcatalog(1M runs) found nothing.Decisions
Decisions 1 to 6 were decided by the maintainer on 2026-10-30.
Nightly with targets that already fail. Six of the eight new targets (fmp4, mkv, ts, cmaf, annexb, flv) fail within a minute on the findings above, and libFuzzer stops at the first failure, so until the findings are fixed those nightly jobs stay red and find nothing new.
catalogandamf0in the nightly matrix, adding each other target once its finding is fixed.DEFLATE encoder per catalog update. Is a fresh ~200 KB encoder per catalog or HLS republish a bug, or an accepted cost?
cache::Track) and billing it to the pool while it waits. That is a change to moq-net's memory accounting on every compressed track, not a small fix to bundle into a fuzz PR.Third-party parser bugs.
sgpdfirst, upstream in kixelated/mp4-atom, without cutting a release, and pin moq to it once merged. Opened kixelated/mp4-atom#264. moq stays on 0.16.2 until Maybe close on disconnected element. #264 merges and a release is cut. A waiting quest tracks that. mp4-atom 0.17.0 already fixes theirefoverflow (Validate nestedirefreference sizes kixelated/mp4-atom#219).quest/m1/fuzz-findings, whose goal is a clean nightly run of every media target (see below).mp4-atom
iloc(mp4-atom-iloc.md).webm-iterable panic and h264/h265 SPS overflows (
mkv-block-header.md,sps-overflow.md).h264-reader, but moq-mux parses the H.264 SPS withh264-parser(rs/moq-mux/src/codec/h264/mod.rs:169) and usesh264-readeronly for TS video timing, so the quest says "the crates moq-mux uses now". If you meant a move toh264-reader, that needs a quest edit.Merge fix: reject sgpd entry counts past the box kixelated/mp4-atom#264 (
sgpdcount bound).06a532aa. mp4-atom'srelease.ymlruns release-plz on push, butCargo.tomlis at the already-published 0.17.0, so nothing was published.mp4-atom-bump.mdnow waits on a 0.17.x release containing Maybe close on disconnected element. #264.Where the fMP4 parse-amplification fixes land (decided by the maintainer on 2026-11-03). The measurement main added to the quest found 116 bytes planning 584 MB, an 8 MiB
moofpeaking at 667 MB, a 32 MiBmoofparsing 369 MB before its refusal, and quadratic CPU in amoov'strakcount.The quest is now
quest/m1/fmp4-metadata-limits.md, ranked in m1 after the media fuzz findings epic.quest mvrefused the move: "refusing to move claimed quests: quest/m2/media-fuzz (feat(fuzz): fuzz the media parsers untrusted publishers reach #7313); their branches would no longer name them, so move them once the work merges or the claim is released". The move was done withgit mvand the links rewritten by hand. This branch now deletesmedia-fuzz.md, which finishes it, andquest check --base origin/mainaccepts that. The new quest sets the limit values itself, rather than settling them with the m2import-metadata-policy.md, so m1 work doesn't wait on m2.Quests added
Under
quest/m1/fuzz-findings/README.md, listed in m1's Required. I used a new epic instead ofm1/security-audit, whose Goal is one external audit's findings; this epic's Goal is the nightly jobs going green.mp4-atom-bump.md: waiting on a 0.17.x release containing fix: reject sgpd entry counts past the box kixelated/mp4-atom#264 (merged), then the move off 0.16.2 and its breaking changes.mp4-atom-iloc.md: zero-sizeilocextents (31 bytes allocate 6 MB). Fixed in mp4-atom by rejecting zero-size extents after an item's first (Decision 4).: deleted in the 2026-11-01 base merge. moq-dev/moq#7234 already replaced webm-iterable's block parsing with moq-mux's own header reader, so the quest's reproducer is committed here as a regression instead.mkv-block-header.mdts-step-scan.md:Ahead::scan'sscan < at.sps-overflow.md: the h264-parser and scuffle-h265/expgolomb overflows, validated in moq-mux before the crates run and reported upstream (Decision 5). Building the reproducers showed that the scuffle-h265 scaling-list underflow also panics in release builds, on the wrapped index.catalog-encoder.md: the DEFLATE encoder per update (Decision 2).quest/m1/fmp4-metadata-limits.md(not in the epic): the three fMP4 parse-amplification fixes, with the measured numbers (Decision 7).Choices made without asking
moq-mux-fuzz,moq-rtmp-fuzz) to match the existingrs/<crate>/fuzzlayout thatsh/rs/fuzz.shresolves. hang catalog parsing is fuzzed from moq-mux, which driveshang::catalog::Consumer.rs/moq-mux/src/fuzz.rs. If a regression input trips the bound once,runmeasures it a second time, because lazy statics allocate once per process.h265::fixturesandcatalog::to_msfcrate-internally under thefuzzfeature, for seeds.regressions/, because each would fail the merge gate until its fix lands. They go in with their fixes. The one exception isregressions/mkv/short-block-header, whose fix landed on main first.Checks
On
8234e5b410(main atf68512665e). Since then, the head changed only by the quest move and a clean merge of main'sbe39188000(moq-net, merged without conflict). Onc1b09990da,quest check --base origin/mainreports 558 documents ok.quest gate just check origin/main: every module passed excepttest rs, and the flake check was skipped (this host's Nix builder cannot create user namespaces).test rsstopped at the first failure each run, in tests this PR does not touch:moq-c test::raw_track_subscription_options_and_updatetwice, which passes 6 of 6 times alone on both this branch and main and is plausibly what open moq-dev/moq#7385 fixes, and then amoq-relayuring test that hit the host'sRLIMIT_MEMLOCK.just rs test --workspace --no-fail-fast(excluding the fuzz crates): 11045 passed, and the 11 failures are allmoq-uringintegration tests failing io_uring setup on the same sharedRLIMIT_MEMLOCK(8 MiB). This includes both crates'fuzz::tests::{seeds_are_accepted,regressions}; moq-mux and moq-rtmp alone pass 2234 of 2234.quest check --base origin/main: 555 documents ok.just rs fuzzpath (nightly toolchain plus ASan) has still not run on this branch.Base merge (2026-11-02)
The PR conflicted with main from 2026-11-01 until now. Auto-merge was disabled before pushing, and the maintainer requeues the reviewed head. Merged
origin/mainthree times (c107623a9dat41f60ec46e,8234e5b410atf68512665e, thenc1b09990daatbe39188000, which merged cleanly):quest/m2/media-fuzz.md(the only conflict). moq-dev/moq#7468 added a requirement: record the peak allocation for a maximalmoov, a maximalmoofof 4-byte entries, and a field-lessmoofat the sample bound, then fix each or accept it with the number. The fuzz targets cannot measure this, because libFuzzer inputs stay a few KiB, far below the 32 MiB bound. So I measured it by hand: a throwaway test fed each input throughfmp4::Import::decode, with moq-mux's counting allocator extended to track peak live bytes. The test is not committed.truns declaring 2^21 samples (MAX_FRAGMENT_SAMPLES), then anmdatheadermdatbyte)trunentries, then anmdatheaderMAX_HELD_BYTESTooManySamplesonly after mp4-atom parses itmoovof repeated audiotraks (477 B each) at 0.24, 0.48, and 0.95 MBThe fixes became
quest/m1/fmp4-metadata-limits.md(Decision 7), linked fromimport-metadata-policy.mdin place ofmedia-fuzz.md. This branch's sentence in that quest, which claimed the fuzz targets measure this amplification, was dropped..config/hakari.toml: moq-dev/moq#6985 added the workspace-hack.moq-mux-fuzzandmoq-rtmp-fuzzjoinmoq-net-fuzzandmoq-quic-fuzzintraversal-excludes;cargo hakari generate --diffandsh/rs/hakari.pypass.sps-overflow.md: moq-dev/moq#7398 moved H.264 SPS parsing from h264-parser to h264-reader. The quest now says so, and points the H.264 half atsps-fuzz-h264-reader.md.mp4-atom-bump.md: its prose wait for an mp4-atom 0.17.x release moved into a## Waitingsection, to match quest's new format. crates.io's latest is still 0.17.0.The fuzz bodies compile unchanged against main's moq-mux (fMP4 refusals that stay terminal, MKV leaf bounds, h264-reader).
Base merge (2026-11-01)
quest landejectedbb2bcbeb8cwith a conflict, and the optionalQuestcheck failed on links to quests main had finished. Mergedorigin/maintwice (206a5fccb9atd8288e6934, thenc3eccc4603atd52d55ac69):quest/m1/security-audit/fmp4-stream-mdat.md(moq-dev/moq#7232),quest/m1/mkv-truncated-finish.md(moq-dev/moq#7608), and thequest/m1/security-auditepic (moq-dev/moq#7611). This branch had only edited their links tomedia-fuzz.md.fuzz-findings/README.mddropped its Related links to the finished MKV and fMP4 streaming quests.block_header(rs/moq-mux/src/container/mkv/import.rs), so themkvreproducer (u6OBgA==) now fails withInvalidSimpleBlock. Committed it asrs/moq-mux/fuzz/regressions/mkv/short-block-headerand deletedmkv-block-header.mdwithquest rm.quest/m2/import-metadata-policy.mdandquest/m2/sps-fuzz-h264-reader.mdstopped linkingmedia-fuzz.md: the first names the fuzz targets in its Plan, and the second drops it from Required, since this PR finishes it.da4ac94df3: an earlier mechanical merge left a 153-character line insh/rs/select.test.py, which failedcheck python. It now builds the--exclude *-fuzzlist from aFUZZ_EXCLUDESconstant.quest/m1/security-audit/README.mdwas valid, ande29bc0d820restored the link. The second merge then took main's deletion of that whole epic, so the file no longer exists.Base merge (2026-10-31)
quest landejectedbd91a2d749with a conflict in.github/workflows/nightly.yml. Mergedorigin/main(a1169021b6) in65647df8b7:nightly.yml: main added moq-net'singestandrequesttargets to the fuzz matrix on the same line; resolved by hand to the union. Target names remain unique acrossrs/*/fuzz/fuzz_targets/.irefoverflow in the findings table.c03b3088fcrewritesmp4-atom-bump.mdto match: it still waits on a 0.17.x release with fix: reject sgpd entry counts past the box kixelated/mp4-atom#264 (crates.io's latest is 0.17.0), but no longer carries 0.17's breaking changes, so it is now[XS].quest/m2/wire-fuzz-coverage.md, linking themedia-fuzz.mdquest this PR deletes.6bcdc65970points that link at thequest/m1/fuzz-findingsepic.Review
quest review, gpt-6.1-sol, medium effort, session01a19fab.8234e5b410(both 2026-11-02 base merges, the hakari excludes, and the quest rewrites; basef68512665e): 0 findings.c1b09990da(the quest move to m1, Decision 7, and the clean merge of main atbe39188000; basebe39188000): 0 findings.c3eccc4603, raised nothing new.(Written by Claude Opus 5.5)
🤖 Generated with Claude Code