Skip to content

ci(rs): lint benches and examples in the PR gate - #7786

Merged
kixelated merged 4 commits into
mainfrom
fix/clippy-benches
Nov 4, 2026
Merged

kixelated merged 4 commits into
mainfrom
fix/clippy-benches

Conversation

@kixelated

@kixelated kixelated commented Nov 3, 2026 •

Copy link
Copy Markdown
Collaborator

Problem

cargo clippy --all-targets fails on main, with default features too: the moq-pattern matching bench added in moq-dev/moq#7536 matches a varint size with ..64 and then ..16384, and clippy::match_overlapping_arm rejects the overlap. The PR gate didn't catch it. The gate runs clippy through the --tests nextest build (moq-dev/moq#5531), and that build never compiles bench or example targets. Only Nightly's features job lints them, and it runs after the merge. sh/rs/select.sh still selected a crate when its benches/ changed "for its lint", but in practice nothing linted the bench.

Approach

  • Bench fix: varint buckets a QUIC varint's encoded size: 1 byte below 64, 2 bytes below 16384, otherwise 4 bytes. The arms are now 0..64 and 64..16384, which are disjoint and return the same sizes.
  • Targets: sh/rs/select.sh's test action (what just check and the land gate run) adds --benches --examples to the one test build. clippy-driver is already the workspace wrapper for that compile, so their lints now fail the PR. nextest's existing default-filter = "not kind(bench)" still keeps the benches from running. The examples build as test-harness binaries with no tests.
  • Required features: Cargo skips a bench or example whose required-features are off, without a warning. A bench-only change selects its crate alone, so no dependent turns them on either. The test action therefore reads every bench and example required-features of the crates in the build from cargo metadata and adds them to the existing capture/play/webrtc/io-uring feature picks, without duplicates. On a workspace run that adds moq-quic/fuzz, moq-stats/bench, moq-net/{async,fuzz}, moq-relay/test-support, moq-mux/fuzz, moq-rtmp/fuzz, moq-uring/quic, moq-audio/bus, and the moq-tokio/qmux example features. All of them are additive: no library has a not(feature = ...) arm on them, and the test count is unchanged. All of them build wherever the gate runs. moq-uring compiles to nothing off Linux, and moq-audio/capture was already on. So no target is left out.
  • Recipes: just rs test keeps --tests. Nightly and the docs call it for targeted --release benchmark runs, so adding targets there would build release benches and examples for nothing. just rs check-test is unchanged too: it forwards the same args to cargo doc, which has no --benches, and nothing in CI calls it.
  • Comments in rs/justfile, sh/rs/select.sh, and .config/nextest.toml now describe the new target set. sh/rs/select.test.py checks the workspace feature list and that a bench-only change in moq-stats, moq-net, moq-relay, or moq-quic turns on that bench's required feature.

Cost

No new dependencies compile. criterion (through bench-harness) is already a dev-dependency, so the tests build already compiled it. The extra work is the selected crates' bench and example binaries, plus whatever library code their required features add.

Both crates were measured after touching src/lib.rs, two runs each. The host had 24 cores at load ~25 from other agents, so the wall times are noisy; CPU (user+sys) is the steadier number.

Crate Build Wall CPU
moq-mux (9 benches, 2 examples) --tests (main) 52.2 s, 43.5 s 102.9 s, 97.9 s
+ --benches --examples 80.7 s, 45.8 s 145.5 s, 118.0 s
moq-net (bench-only change) --benches --examples, default features: 3 benches 61.9 s, 54.6 s 172.4 s, 177.1 s
+ --features moq-net/async moq-net/fuzz: all ~23 benches 65.9 s, 54.6 s 217.8 s, 197.0 s

Against main, moq-mux's edit loop costs about +20–45% CPU, including its ~18 s test run. The required features add another +11–26% CPU for moq-net. moq-net's test count stays at 3,435 either way.

Impact

  • Public API and wire: none.
  • CI: a PR now fails when it introduces a clippy finding in any bench or example of the crates it selects, including targets behind required-features. Nightly no longer has to report it after the merge. Each gate run also compiles those binaries and their features (see Cost).

Alternatives

  • --all-targets in the just rs test recipe: fewer edited lines, but every targeted --release benchmark run (Nightly, bench/README.md) and the macOS test runs would also build every bench and example. It would still skip the required-features targets.
  • --all-features for the selected crates: this would also turn on backend features no bench or example needs, such as GPU encoders, vaapi, and aws-lc-rs beside ring. Nightly's features job already compiles that set. In the gate it would add compile time and run tests under feature shapes the gate doesn't test today. Each target's own required-features is the narrowest complete set.
  • A separate cargo clippy --benches --examples pass: check mode can't reuse the test build's artifacts, so it would re-check the whole dependency graph. That is the second compile the recipe comments avoid.
  • Fix only the bench: this kind of failure would reach main again.

Validation

  • With the gate change and the original bench, just rs test -p moq-pattern --benches --examples fails with match_overlapping_arm on benches/matching.rs. With the fix, it passes.
  • cargo clippy --keep-going --workspace --all-targets (excluding the fuzz crates), with the gate's device features plus every target's required features, found nothing else.
  • just check origin/main on 415e69bd01 passed every selected module: rs, biome, markdown, gh, python, shell, and the text checks. The rs/justfile change selects the whole workspace. test rs ran 11,638 tests (132 skipped) and built every bench and example, including producer_tick, the six moq-quic benches, and moq-audio's bus. It took 531 s wall with a warm cache at host load ~25.
  • python3 sh/rs/select.test.py: 24 tests pass on 1a135b069e.
  • No Nightly or full just gate was run or claimed.

Review

Codex, gpt-6.1-sol, medium effort, local quest review:

  • Round 1 reviewed head b45ec9d76183037d20499a553bebaaf0ab4140ba against origin/main (40c4eb0b428d5579007cae86dbc7c90aff6bccec): no findings.
  • Round 2 reviewed head 415e69bd0188e5b2d60206b08301960d39f39661 against origin/main (b23b31e98edc1bdd3b511f4052cf0391022b25c2): no findings.

Pullfrog's thread on sh/rs/select.sh was fixed in 415e69bd01: benches behind required-features were silently skipped. The thread is resolved.

Pullfrog's thread on sh/rs/select.test.py was fixed in 1a135b069e, a test-only change after round 2. test_a_bench_selects_only_its_crate had lost its check that a moq-stats library change selects moq-relay. The new required-features test ran that check instead, using its loop's leftover bench variable rather than the moq-stats bench. The two assertions are back in their own test, and select.test.py passes. The thread is resolved. No new review round was run.

(Written by Claude Opus 5.5)

🤖 Generated with Claude Code

kixelated and others added 2 commits November 3, 2026 15:30
The moq-pattern matching bench's varint match had overlapping ranges
(`..64`, `..16384`), which `cargo clippy --all-targets` rejects as
`match_overlapping_arm`. The gate's clippy runs through the `--tests`
nextest build, which never compiles bench or example targets, so only
Nightly could see it.

Make the varint ranges disjoint, and have the gate's test build
(sh/rs/select.sh `test` and `just rs check-test`) also compile
`--benches --examples` so clippy-driver lints them; nextest's default
filter still leaves the benches unrun.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@coderabbitai

coderabbitai Bot commented Nov 3, 2026 •

Copy link
Copy Markdown
Contributor

Warning

Review limit reached

You've used all free OSS reviews for now. Wait for the free limit to reset to keep reviewing this public repository.

Next included review available in 11 minutes.

Check out review usage here.

View limit details

Limit details: You’ve used all 4 included reviews currently available.

Learn how review limits work.

Review configuration:

Run configuration
  • Source mode: Auto
  • Repository: Unavailable
  • Branch: Unavailable
  • File: No configuration file from this repository applied
  • Commit: Unavailable
  • Layers: Organization UI
  • Review profile: CHILL
  • Plan: Advanced
  • Run ID: a74d11b6-5648-4dbb-9d15-bb65c316eece
Commits

Reviewing files that changed from the base of the PR and between 40c4eb0 and 1a135b0.

Files selected for processing: 5
  • .config/nextest.toml
  • rs/justfile
  • rs/moq-pattern/benches/matching.rs
  • sh/rs/select.sh
  • sh/rs/select.test.py
  • Autofix · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

Share

@kixelated kixelated left a comment

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Automated review by review (OpenAI)

Reviewed head b45ec9d.

No actionable findings. The disjoint varint arms preserve the intended buckets, and both selected test builds and check-test now include bench/example targets while the nextest filter excludes benchmark execution. The selector tests reflect the new flags, and keeping ordinary targeted rs test unchanged limits unnecessary release builds.

Verification: GitHub-only static review of the gate, selector, nextest configuration and changed bench. I did not run CI, tests, or the timing measurements.

@pullfrog pullfrog Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ℹ️ One coverage gap worth confirming; the rest looks right.

Reviewed changes

  • Gate builds bench/example targets: sh/rs/select.sh's test action now appends --benches --examples, so the clippy-driver-wrapped nextest build lints them in PRs instead of only in Nightly.
  • Bench lint fix: varint's overlapping match arms become disjoint 0..64/64..16384, preserving the 1/2/4-byte buckets that match_overlapping_arm rejected.
  • Targeted test recipe unchanged: just rs test keeps --tests; only check-test and the selector's test action add the new targets, so --release benchmark runs don't build them.
  • Docs and tests updated: .config/nextest.toml and rs/justfile comments plus sh/rs/select.test.py expectations match the new flags.

Pullfrog  | Fix all ➔ | Fix 👍s ➔ | View workflow run | Using deepseek-v4.1-flash-1017 (free via Pullfrog for OSS) | 𝕏

Comment thread sh/rs/select.sh Resolved
Cargo skips a bench or example whose `required-features` are off without
a word, so `--benches --examples` still left moq-quic's benches,
moq-stats' `producer_tick`, and on a bench-only change moq-net's `fuzz`
and moq-relay's `session` benches unlinted. sh/rs/select.sh's `test`
action now reads the built crates' target `required-features` from
`cargo metadata` and turns them on in the one test build, beside the
device picks. They are all additive.

`just rs check-test` drops `--benches --examples`: it forwards the same
args to `cargo doc`, which has no `--benches`, so it could not carry the
features, and nothing in CI runs it.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

@kixelated kixelated left a comment

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Automated review by review (OpenAI)

Reviewed head 415e69b, focusing on the update since b45ec9d.

No new actionable finding. The test selector now gathers required features for selected bench and example targets, including the isolated crates cited in the earlier review, addressing the gap in gate coverage. The ordinary check-test recipe no longer includes those targets, but the gate uses the selector's test action.

Verification: GitHub-only static inspection of the selector, manifests and tests. I did not run the gate, tests or clippy.

(Written by review, OpenAI)

@pullfrog pullfrog Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ℹ️ No critical issues — one minor test-organization suggestion inline.

Reviewed changes

Reviewed the delta since the prior pullfrog review (b45ec9d), which adds required-feature coverage to the gate.

  • Bench/example required-features enabled: sh/rs/select.sh's test action now reads every bench and example required-features from cargo metadata for the crates in the build and appends them to the existing device/runtime feature picks.
  • Build membership tracked: building carries the selected crates plus the media packages the action adds, so targets' features stay scoped to what the build holds.
  • Tests extended: sh/rs/select.test.py derives the workspace feature list from metadata and asserts four bench-only selections enable their required feature; the metadata fixture expectation gains --benches --examples.
  • Comment updated: rs/justfile now describes the required-features behavior.

Pullfrog  | Fix all ➔ | Fix 👍s ➔ | View workflow run | Using deepseek-v4.1-flash-1017 (free via Pullfrog for OSS) | 𝕏

Comment thread sh/rs/select.test.py Resolved
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

@kixelated kixelated left a comment

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Automated review by review (OpenAI)

Reviewed head: 1a135b0.

The test-only update restores the moq-stats library-change dependent-selection assertion to its own test with the intended bench path, rather than inheriting the feature loop's last path. No additional actionable finding in this delta. Verification: static GitHub inspection; I did not run the selector tests, gate, or clippy.

(Written by review (OpenAI))

@pullfrog pullfrog Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

✅ No new issues found.

Reviewed changes

Reviewed the delta since the prior pullfrog review (415e69b): a test-only follow-up in sh/rs/select.test.py.

  • Restored the dependent-propagation check: the moq-stats bench plus rs/moq-stats/src/lib.rs selection assert that moq-relay is pulled in now lives back in test_a_bench_selects_only_its_crate, where bench is the moq-stats target it was written for.
  • Cleaned up the required-features test: test_a_bench_alone_enables_its_required_features no longer reuses the loop's leftover bench variable after its loop.

python3 sh/rs/select.test.py runs 24 tests; the only failures are the known env-only ones (sudo no-new-privileges / missing just). The reorganized tests pass.

Pullfrog  | View workflow run | Using deepseek-v4.1-flash-1017 (free via Pullfrog for OSS) | 𝕏

@kixelated
kixelated added this pull request to the merge queue Nov 4, 2026
Merged via the queue into main with commit ec12a42 Nov 4, 2026
8 of 9 checks passed
@kixelated
kixelated deleted the fix/clippy-benches branch November 4, 2026 02:52
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant