[kixelated]

Problem

The relay's instructions per delivered frame grew after moq-dev/moq#5926. The quest asks for each rise to be found, attributed, and fixed when accidental. The first round bisected the relay counts: the two large accidental rises were already gone from main, and three decisions were left for the maintainer.

Decisions

Decided 2026-11-03 (maintainer):

1. Cursor::probe on every peek (moq-dev/moq#6171, about 8.6k instructions per frame):
   • ✅ Add a fast path in this quest that skips the work while nothing is missable.
   • Accept it as the cost of relay backfill.
2. The cursor's window, drift, and live-edge reads (moq-dev/moq#5929), and the 2-4k steps from compressed tracks, ranged reads, in-tree qmux, signed priorities, public ingest, and receive leases:
   • Accept them as feature costs.
   • ✅ Investigate each: profile each step's added instructions, fix what can be cut without changing behavior, and attribute what can't with the evidence.
3. kio's task and waiter growth (about 11k) and the lite publisher's serving and close polling (about 6k):
   • ✅ Profile them in this quest, then fix or attribute them before closing.
   • Hand them to the performance audit.

Decided 2026-11-03 (maintainer), after the attribution:

1. Costs measured and not cut:

   • ✅ Accept the flat ones (per-reader ranges' remainder, ranged reads, public ingest, compressed tracks, moq-dev/moq#7052, bounded reads), recorded with their numbers in the release-perf epic. Keep moq-dev/moq#6995's and moq-dev/moq#7542's session cost in the quest for a re-measure against this branch.
   • Accept all of them now.
   • Keep investigating all of them.

2. The waiter's list records:

   • ✅ A follow-up at m0, framed as a correctness bug: raising the limit from 8 to 16 makes two moq-net integration tests fail, so a wake is missed or relied on by accident. Planned here as a waiter's record limit changes no behavior, with the two tests and how to reproduce.
   • Leave the limit at 8.

3. group::Consumer moves:

   • ✅ A follow-up in the release-perf epic, measured with just bench-fanout: a read moves a small group consumer.
   • Leave it.

4. Land this PR now as a checkpoint, or keep it open until the quest's remaining measurements are done:

   • ✅ Land now. The quest stays open with what remains: attributing in-tree qmux and receive leases on a quiet host, re-measuring moq-dev/moq#6995's and moq-dev/moq#7542's session cost against main, and re-pairing main against the old endpoint.
   • Keep it open and finish the measurements here first.

No existing quest overlapped either follow-up. They link the related task-capacity (a task slot holds a whole waiter) and relay-allocs (boxing must not add a per-group allocation) quests.

Approved in follow-up planning the same day: give the driver a load-independent mode for the chat fan-out and use it for the feature-cost investigation, with the quiet test hosts available for relay runs.

Cursor::probe is the relay's check for groups a reader has missed. Each time a subscriber asks for its next group, the probe looks at what arrived on the track since its last look and decides whether any group the reader still wants has left the cache without the live feed delivering it (evicted, skipped, or from before the feed started), and asks upstream to fill those. A live reader that takes each group as it arrives never has such a gap, yet it paid for the whole check on every group: locking the subscription and the track's shared state, cloning the track's fills, walking the arrival ledger, checking each new group for holes, and running the ask-and-answer loop. Each time the cache aged out an old group, the eviction also landed with every reader, all of which had already taken it, and forced each one to re-run the full check.

What changed

A load-independent driver mode, just bench-fanout, counts the chat fan-out per delivered group on one pinned thread, for any list of revisions:

• model: one finished group a cycle through the track model to every subscriber. It builds against every revision back to cdff3c8f95.
• session: a publisher session into a relay origin and a viewer session out of it per subscriber, over the in-memory transport. It builds back to 2026-11-01.

Each count is user instructions from perf stat for a run of many cycles less a run of zero, the median of three. Host load doesn't move it; each process's hash seeds move it by up to 0.5%. Callgrind attribution, and a comparison of two revisions by function, are kept on request.

Fixes, each its own commit, counted together in one run at 512 subscribers per delivered group:

Change	Model	Session
main	13,300	120,948
A live reader's probe skips the fills while it misses nothing	10,477 (-2,823)	114,139 (-6,809)
A window with one set of runs reads without growing across two	10,126 (-351)	113,133 (-1,007)
This branch	10,126 (-23.9%)	113,133 (-6.5%)

After merging main at 619df6e1b3, against it: model 13,368 to 10,119 (-24.3%), session 121,114 to 113,143 (-6.6%).

• The probe fast path judges a group the arrival walk serves without the hole, backfill, and ledger passes. It skips the subscription, fills, and answers when nothing is asked or owed, and starts each reader's eviction inbox above what it handed out. A priority change still reaches any fill the reader joined. All 3,348 moq-net tests pass; one test that asserted inbox contents now expects an evicted group the reader took to stay out of its inbox.
• A window's reads used to grow each run across its fixed ranges and its time ends' layer, though most windows have no layer.
• Review found that the fast path for a live arrival judged from a resumed copy's live floor, which moq-dev/moq#7708 had just stopped doing: an unlimited reader's history below that floor could be forgotten and, once evicted, never asked again. It now judges from the reader's floor, and a regression test fails without the fix.
• Tried and reverted: a kio waiter that remembers 16 lists instead of 8. A relay session's control and serving tasks park on more than 8, so every list past the eighth takes a duplicate entry on every poll; 16 cut the session by 3.9% (12: +1.6% over that, 32: +1.1%). But two integration tests then fail (errored_end::an_errored_end_keeps_its_history_across_a_hop and quiet_catalog::a_quiet_catalog_reaches_an_unfloored_relay_reader_when_a_peer_resumes_past_it) and pass with 8. A read whose waiter parks on more lists than it records is retired and walks again; with more records those reads stay parked, and something they depend on doesn't wake them. The revert and its enum boxing are separate commits; the change's own commit message overstated its model effect, measured with frame pointers forced.

On the quiet test host nixos2 (one-minute load at most 4.7), chat over io_uring went from 154.5k [154.1-155.0] instructions per frame on main to 148.9k [148.3-150.8] with the probe and window changes (-3.6%), four rounds each, every cell at 20,000 frames/s with no loss. QUIC, crypto, and syscalls are most of the relay's cost, so the model's -24% shows as a few percent there. quest gate isn't installed on nixos2, so its load and other agents were checked before each run.

Attribution

The report has the full tables. The model path cost 4,285 instructions per group at the old endpoint and 13,300 on main. The features asked about, each against its parent (model, 512 subscribers):

Feature	Change
Per-reader ranges and time anchors, moq-dev/moq#5929	+902
Compressed tracks, moq-dev/moq#5928	+33
A subscriber reads only its ranges' frames, moq-dev/moq#6129	+183
Signed priorities, moq-dev/moq#6142	-72
Public ingest, moq-dev/moq#6111	+200

In-tree qmux (moq-dev/moq#6231) and receive leases (moq-dev/moq#5878) are QUIC-level, which neither scenario reaches. A relay run pairing each with its parent on nixos2 got one round (qmux 116k to 122k, leases 130k to 132k per frame) before other agents started fuzzing and compiling there; it was stopped, and neither is attributed yet.

Most of the model's growth came later, in a few steps found exactly: moq-dev/moq#6171 +4,264 and moq-dev/moq#7339 +1,154, which the probe fast path recovers most of; then moq-dev/moq#7542 +876 (session +3,054), moq-dev/moq#6995 +558 (session +3,290), and moq-dev/moq#7052 +365.

For decision 3, the session path's largest remaining self costs are moves of large values (about 9.4k per group: group::Consumer is 280 bytes, moved by value through poll_peek_group, take, and the viewer's serve loop) and waiter registration (about 7.2k, with the record limit above). The rest of the lite publisher's serving is the per-group uni stream's open, write, and close, spread across small costs.

Impact

• Public API and wire: no changes.
• Benchmark tooling: just bench-fanout, bench/fanout/, bench/fanout.py; the instructions mode builds BENCH_BUILD_JOBS at a time.

Review

Reviewer: Codex (gpt-6.1-sol, medium effort) via quest review --base origin/main, one session since the design change (--new).

• Round 1, head af83ff87fc, 1 finding, fixed in 0e9b7ae6bb:
  • rs/moq-net/src/model/track/miss.rs:1000: the live-arrival fast path judged from a resumed copy's live floor, losing an unlimited reader's history below it once evicted. Fixed by judging from the reader's floor, as the full judgement does since moq-dev/moq#7708. The regression test history_below_a_resumed_live_floor_is_refilled_after_a_live_arrival fails with the old floor ("asked []") and passes now.
• Round 2, head 0e9b7ae6bb: no findings.
• Round 3, head 77da888fe4 (docs and evidence since round 2): no findings.
• Round 4, head 21b6ab173a (a merge of main, the two follow-up quests, the epic's accepted costs, and the recorded decisions): no findings. It was the design checkpoint's fourth round; the approach stands, since nothing was left to fix.

Head 249b3687fc adds only a conflict-free merge of main since round 4, which the guide exempts from a new round, and this PR body records the checkpoint decision; no new round ran.

Checks

• quest gate just check origin/main on 249b3687fc, after merging main, passed every module but two. check carla is main's own break since moq-dev/moq#7691 (demo/carla/sim/moq_carla_sim/publish.py:202 misses the new timestamp argument), which this branch doesn't touch; open moq-dev/moq#7755 fixes it. check flake failed on a temporary directory under .tmp/ that another module created and removed during evaluation; run alone, it is skipped here because the local Nix builder can't create user namespaces. It passed in the full run on 77da888fe4.
• quest check --base origin/main passes (567 documents after the merge).
• moq-net, kio, and moq-stats: 3,634 tests pass, and clippy is clean with warnings denied, on 0e9b7ae6bb; no code changed since except main's merge, which just check covered.
• bench/fanout.test.py and the rest of the bench tests, Ruff, and shellcheck pass. Both driver modes ran end to end.
• Measurements: every local run went through quest gate, at one-minute loads up to 40 on this host; the fan-out counts don't depend on load. Relay runs were on nixos2, which was idle when its first run started; a second run there was stopped when other agents started work. kernel.perf_event_max_sample_rate was restored after each run on both hosts.

Follow-ups

• Planned in this PR: a waiter's record limit changes no behavior (m0) and a read moves a small group consumer (release-perf).
• mbx's cache evicts worktree target directories under disk pressure, which broke two long sweeps here; bench-fanout gained BENCH_FANOUT_TARGET to build outside it.

(Written by Claude Opus 5.5)

🤖 Generated with Claude Code

[kixelated]

Design change: per the maintainer's 2026-11-03 decisions, this PR now changes production code as well as measuring. It adds a fast path to Cursor::probe for readers with nothing missable, a single-set shortcut for window reads, and a fix keeping history below a resumed live floor in that fast path. It also adds just bench-fanout, a load-independent per-group count of the chat fan-out (model and in-process session), used to attribute each feature step exactly. A larger kio waiter record (8 to 16 lists) was tried and reverted: it made two integration tests miss a wake. Local review restarted with quest review --new for this design.

(Written by Claude Opus 5.5)

[coderabbitai]

Warning

Review limit reached

You've used all free OSS reviews for now. Wait for the free limit to reset to keep reviewing this public repository.

Next included review available in 18 minutes.

Check out review usage here.

View limit details

Limit details: You’ve used all 4 included reviews currently available.

Learn how review limits work.

Review configuration:

Run configuration

• Source mode: Auto
• Repository: Unavailable
• Branch: Unavailable
• File: No configuration file from this repository applied
• Commit: Unavailable
• Layers: Organization UI
• Review profile: CHILL
• Plan: Advanced
• Run ID: 24b2b7e0-dc71-465c-983a-74cd42dc6f5e

Commits

Reviewing files that changed from the base of the PR and between 7e4ed93 and 249b368.

Files selected for processing: 25

• .github/justfile
• .github/relay-cpu-bisect.json
• .github/relay-cpu-bisect.md
• bench/README.md
• bench/fanout.py
• bench/fanout.test.py
• bench/fanout/model/Cargo.toml
• bench/fanout/model/src/main.rs
• bench/fanout/session/Cargo.toml
• bench/fanout/session/src/main.rs
• bench/instructions.py
• bench/metadata.py
• bench/profile.py
• bench/profile.test.py
• bench/relay.sh
• bench/run.sh
• justfile
• quest/m0/README.md
• quest/m0/kio-waiter-records.md
• quest/m1/release-perf/README.md
• quest/m1/release-perf/group-consumer-size.md
• quest/m1/release-perf/relay-cpu-bisect.md
• rs/moq-net/src/model/subscription/runs.rs
• rs/moq-net/src/model/track.rs
• rs/moq-net/src/model/track/miss.rs

• Autofix · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

Share

• X
• Mastodon
• Reddit
• LinkedIn

[kixelated]

Automated review by review (OpenAI)

Reviewed head 249b368.

No actionable findings in this pass. The probe's fast path is guarded against changed budgets, windows, drops, evictions, deferred gaps and refusals; active fills still receive updated preferences and answers. The delivered-prefix eviction narrowing and one-layer window shortcut preserve the intended live-reader behavior, and the resumed-floor regression covers the earlier history concern. The bounded fast path is a reasonable direction for the measured cost.

Verification: GitHub-only static review of the model changes and tests. I did not run tests, benchmarks, or reproduce the reported measurements.

[pullfrog]

✅ No new issues found.

Reviewed changes

• Live-reader probe fast path — Cursor::probe now reads the subscription's asking lazily, clones the track's fills only when something is asked, and skips the ask/resolve loop while no fill is joined and no gap is deferred; Misses::judge_arrivals handles the arrival-only transition, marking the new walked groups judged and trimming below the reader's floor (the resumed-live-floor fix from 0e9b7ae6b).
• Eviction inbox floor — inbox_floor raises each reader's inbox span to its contiguous delivered floor, so evictions of groups it already took no longer land and force a full re-judgement; the full judge and the fast path share it.
• Single-set window reads — Layered::only lets first/last read the one populated run set directly instead of growing across two.
• Fan-out and instructions tooling — just bench-fanout with the bench/fanout/{model,session} harnesses, the instructions profile mode with rotated multi-revision relay runs, and the report/evidence under .github/.
• Quest records — the maintainer's 2026-11-03 decisions, the accepted feature costs, and the waiter/group-consumer follow-up quests.

I traced the fast path against the full judge for arrivals, evictions, ledger changes, deferred/refused gaps, and the is_idle skip, and could not produce a divergent reader-visible outcome; the new regression test exercises the resumed-floor case round 1 found, and the remaining test change reflects the intended inbox semantics. No public API or wire changes.

  ^{｜ View workflow run ｜ Using deepseek-v4.1-flash-1017 (free via Pullfrog for OSS) ｜ 𝕏}

[pullfrog]

This run was cancelled before it finished.

  ^{｜ View workflow run ｜ via Pullfrog}