[kixelated]

Problem

A lite-07 SUBSCRIBE fits about 32K ranges in its 64 KiB message. After moq-dev/moq#6995, the Range count cost quest listed the walks that still grew with range count on each poll, and a peer can multiply each one by the number of ranges it names.

This PR finishes that quest. Every path in the ranges bench now costs about the same per step at 16,384 ranges as at 1, and the quest file is deleted.

Approach

Earlier rounds, kept as they were:

• Lite-07 frame bounds (NET-1). Frames in lite/publisher.rs is a BTreeMap keyed by group, with the parts merged as the bounds decode. Each lookup is one search.
• Timed datagram admission. Window keeps an Admission index, four sorted lists for the four pairings of group or time bounds. Left as it was: moq-dev/moq#7575 removes it with stream-track datagrams.
• Time anchors: shared walks. Unsettled time ends whose walks agree share one Walk, holding their times in a BTreeSet. A group starting between them splits them, and walks that converge merge.
• Reverse search over a reader's runs. find_servable searches a sorted Vec of runs by binary search, not rangemap's iterator.

This round (decisions 2 to 4 below):

• Window moves: layered runs (model/subscription/runs.rs). The window keeps its groups as Layered: shared fixed runs (Arc<Runs>) with the walks' runs layered over them (also an Arc). Each query (contains, first or last run reaching into a span, runs from a point) answers from both by binary searches. When a layered run covers many fixed runs, a query grows the run across them in a few searches instead of visiting each one. A move rebuilds only the layer. Settling a time end, and a walk parking or waking (below), rebuild the fixed runs, which happens once per end or per gap event, never per group. Each pass that followed a move now works from the layers:
  • Window::rebuild sets the layer from the walks not parked.
  • Servable::Runs holds the layered set, its cap, and the reader's live index, so update_servable copies two Arcs, and Published compares the fixed runs by reference first.
  • Cursor::admit keeps the admitted set as a layered snapshot (Admitted). When only the layer moved, it owes what the layer gained, but only where the cache holds a group now, the only place next_owed can find one. It also cuts what the layer lost, only from what is owed.
  • Misses::retain and Misses::judge keep the last missable view. When only the layer moved, they forget what it lost and judge what it gained, within what they hold or judge (from the floor up to the newest group, below the cap), instead of clipping every set to the whole window again.
  • covered_by needs nothing: it returns early while a time end is unsettled, so its memo only restarts after a settle.
• Sparse live numbering: a reader's index of its live groups (Live in track.rs). Each Servable::Runs carries an index of the live feed's cached groups inside its fixed runs, shared by every servable built on the same fixed runs. It takes in the arrival order on the reader's own search, one lookup in the fixed runs per arrival. An entry whose slot left or was replaced is dropped when a search meets it, or once stale entries could outnumber the live groups. find_servable searches the index for the fixed runs and leapfrogs only the layer's few runs, short of what the index found. Entries are handed out in batches with the lock released, since an answer reads each group's own state.
• Time ends stuck behind gaps: parked walks (Window::parked). A walk whose settle stopped at a gap below the group its ends fall before, with nothing between that could still start, parks. Until a sequence from above known up to that group arrives or is given up, it reads the same answers. So Holdings::changes reports the runs whose arrivals or drops changed (the arrival ledger's and the drops' change indexes), and a settle wakes only the parked walks a change reaches. The declared start, completion, or a fetched group (which the ledger never sees) wakes them all. Parked walks whose waits overlap are kept together, so the parked groups stay disjoint and one search finds each. Their runs live in the fixed runs, so a moving cap beside thousands of them still rebuilds only the walks not parked.
• Outstanding fills (miss.rs):
  • Expire by successor. Every stretch of a wanted run starting below the first live group after a stretch shares that group as its successor, so one poll_drifted judges all of them. A fresh stretch with no arrival before the next live group is passed in one step, however many runs lie between. A property test checks this against judging each stretch alone.
  • Resolve only the fills that answered. Each join parks on a waiter of its own (JoinWake), which records its key before waking the task the reader last resolved for. resolve looks only at the recorded joins and the new ones, so a reader read by another task since still learns of every answer.
  • judge no longer walks every wanted run to clip what an eviction gave up again: it visits only the runs overlapping what it found.
• ranges_time_gaps bench. Time ranges each ending between two groups of a sparse history, so each waits on a gap of its own, beside one time end past the live edge whose cap moves twice per group.

Tests

New property tests, each comparing the fast path with the walk it replaced, with deterministic generators so a failure replays:

• layered_runs_match_their_union, a_moved_layer_gains_exactly_the_new_groups, inserted_runs_merge_as_a_set, a_run_minus_runs_is_its_gaps: the layered set against the set of its runs merged.
• layered_searches_match_a_scan_of_every_live_group: find_servable through the index and layer against a scan of every live group, in both directions, as groups arrive, start, abort, leave the cache, or arrive again between searches that share one index. It found an early return the suite missed (the index finding group 0 ended the whole search).
• expiring_by_successor_matches_each_stretch_judged_alone: chunked expiry against the old per-stretch judgement, on both clocks, against one run or several.
• a_moved_layer_is_judged_as_the_whole_window_would_be: retain and judge on a moved layer against a second reader handed an unshared copy of the same fixed runs, which judges the whole window each time. Removing either incremental cut makes it fail.
• a_moving_time_cap_owes_what_it_newly_admits: admit owes, of every cached group, exactly what was owed and is still admitted, plus what is newly admitted. Dropping one gained run makes it fail.
• an_answered_fill_wakes_the_task_that_read_last: four fills, one answers. Only its join is looked at again, and the later task wakes.
• shared_walks_match_each_end_walked_alone now exercises parking. Its test holdings report changes as a track does: arrivals and runs given up, never a start. It caught two wrong parking rules while I wrote this (overlapping waits, and a late arrival below the gap). It passes 8,000 rounds on two seeds, with about 69K parks.

Measurements

Criterion's time per step, pinned to P-core 6. "Before" is this PR's previous head, a625ae96e4, run with this round's bench file. Earlier rounds reported some rows in instructions; these are all times, from the same run.

bench	ranges	before	after
ranges_time_cap	1	4.22 µs	4.35 µs
	64	15.6 µs	6.47 µs
	1024	392 µs	8.42 µs
	16384	7.29 ms	8.92 µs
ranges_fills_out	1	3.74 µs	4.05 µs
	64	76.5 µs	4.17 µs
	1024	1.20 ms	4.29 µs
	16384	21.9 ms	4.51 µs
ranges_sparse	1	2.12 µs	2.15 µs
	64	4.18 µs	2.18 µs
	1024	49.8 µs	2.22 µs
	16384	1.11 ms	2.40 µs
ranges_time_gaps	1	5.89 µs	5.20 µs
	64	92.5 µs	5.71 µs
	1024	3.34 ms	6.19 µs
	16384	setup did not finish in 10 minutes	7.17 µs
ranges_time_end	1 / 16384	2.91 / 2.97 µs	2.99 / 3.04 µs
ranges_datagram	1 / 16384	285 / 288 ns	284 / 287 ns
ranges_frames	1 / 16384	1.50 / 44.0 ns	1.50 / 43.7 ns

At one range most rows cost the same, within 3 to 8%. ranges_time_cap steps from 4.4 µs at one range to 6.5 µs at 64: one range is a single run, kept off the heap, and several runs allocate the layered servable on each move. From 64 to 16,384 it is flat up to the binary searches. ranges_frames grows with the depth of its BTreeMap, as in the earlier rounds.

Impact

• Public API: none. fuzz::FrameBounds is only exposed through the hidden fuzz module.
• Wire: none.

Decisions

1. Time anchors: how to stop a settle advancing every unsettled end.
   • ✅ (a) Shared walks (maintainer, 2026-11-01). Done in an earlier round.
   • (b) Index by the waited sequence. Not chosen: it leaves the case the Plan named O(ranges) per group.
   • (c) Charge each range against request flow control. Not chosen: a wire change.
2. Sparse live numbering: how to answer a search in one lookup.
   • ✅ (a) Each reader indexes the live groups inside its window (maintainer, 2026-11-01). Done as Live, indexing the fixed runs, with the layer's few runs leapfrogged.
   • (b) Memoize live_edge per cursor. Not chosen: it leaves successor and arrival.
   • (c) Accept the leapfrog. Not chosen.
   • The maintainer expected the index to also fix time ends stuck behind different gaps. It can't on its own: a stuck walk also reads starts, evictions, drops, and the ledger's folds, which the arrival order doesn't carry. So the walks park, and the ledger's and drops' change indexes wake them (above). Same goal, one more mechanism.
3. Window moves: a move still cost every fixed run.
   • ✅ (a) Layer the walks over shared fixed runs end to end (maintainer, 2026-11-01). Done through Window, Servable/Published, admit, and Misses.
   • (b) Make only Window::rebuild incremental. Not chosen: it leaves the other passes.
   • (c) Accept it. Not chosen.
4. Scope.
   • ✅ (b) Finish the outstanding fills, sparse numbering, and window moves here (maintainer, 2026-11-01), not in follow-up PRs.
   • (a) Land this and do the rest in focused PRs. Not chosen.

The quest is finished and deleted in this PR, so its Plan isn't updated: these records are the paper trail, with git history keeping the quest's last Plan.

Choices made without asking

• What a moved layer owes is cut to the cached groups. admit owes what the layer gained only between the lowest and highest cached sequence. A live group arriving later in that span comes from the arrival walk, as one arriving above it always did, not ahead of it through next_owed. The property test checks every cached group.

• Parked walks drop out of watch_starts and speculative. A parked walk waits on an arrival, never a start, and an arrival changes the track a read parks on anyway.

• A fetched copy wakes every parked walk. Fetched backfill, and a fetched head joining a live group (which can give a group received past frame 0 its start), are never in the arrival ledger. So the track counts them (fetched), and a change in the count wakes all parked walks rather than adding a third change index.

• Rebuilt fixed runs keep the old Arc when unchanged, so a walk that wakes and parks again on the same gap keeps every reader's diff on the layer.

• The test holdings model the track's change reporting, so a starting group reports nothing there either. The parking rules are tested against what the track reports, not what a richer holdings could.

• Every window walk reads the reader's own gone runs, where main used the track's alone when replacing or first placing a window and taking in a lookup's answer. Main's walks redo each settle, so this cost nothing there. A parked walk wouldn't redo it, and could stay parked on a gap already gone for its reader.

Remaining cost, by design

• Settling a time end, and a walk parking or waking on a gap, rebuild the fixed runs: O(fixed runs) once per end or per gap event. A publisher that fills a reader's gaps one group at a time pays that once per such group.
• Placing or replacing the ranges costs them all once, as before.

Merging main

Main landed every PR this one expected to conflict with (moq-dev/moq#7545, moq-dev/moq#7499, moq-dev/moq#7310, moq-dev/moq#7493); moq-dev/moq#7207 closed unmerged. Merged in 01e4c8296b:

• Time-start lookups (moq-dev/moq#7310) on the shared and parked walks. A span looking up its start keeps its own looking list in Window, outside the walks and runs, so lookups() (read on every probe) costs the lookups, not every span. Window::found places the span as fixed runs or as a new walk that walks from where the span reaches, as main's did. Main's three lookup tests run on the branch's window.
• A reader's own gone runs wake its parked walks. Main counts the runs a reader's own fills refused or gave up as gone for its time ends (main's per-reader Seen holdings, renamed ReaderHoldings since it collided with window::Seen). A walk parked on a gap its fill then refuses would never wake, since the track's change indexes don't see it. Misses now records what it newly gives up (take_gone), and the holdings report those runs as changes. Every window walk reads these holdings: a replaced window, a pending one, a settle, and a lookup's answer. None parks on a gap already gone for it. New test: a_time_end_parked_on_a_gap_wakes_when_its_fill_refuses_it, which fails without the wake.
• A handler appearing or leaving wakes every parked walk. looks_up changes which starts count and what is gone, so the track's change marks include it.
• Conflicts in Cargo.toml (both the ranges and backing benches), lite/publisher.rs (main's .parked() subscription), and the probe judgement (the layered retain inside main's single lock). Main's new judge tests take both main's bound and this branch's view.

4147d28035 then merged main again, conflict-free.

fa01c8a663 merged main again, up to 5e327b55f8. Its one conflict was the std import list in lite/publisher.rs: main's moq-dev/moq#7500 added borrow::Cow, and this branch added BTreeMap for Frames, so the merge keeps both. Main's request changes (the track name moved out of the message) touch no code this branch changes. The branch's added and removed lines against main are the same as on the reviewed 01e4c8296b; only context lines differ. 74708ae931 then merged main's quest-only moq-dev/moq#7616, conflict-free.

Checks

On fa01c8a663, the last merge that touches code:

• just check: passed, 9501 tests.
• quest check --base origin/main: passed, also on 74708ae931.

On 4147d28035:

• a_time_end_parked_on_a_gap_wakes_when_its_fill_refuses_it fails with the reader's newly gone runs left out of the changes.
• A spot check of the ranges bench (3 s measurement, P-core 6, loaded host) stays flat in the range count: ranges_time_cap 4.97 / 9.81 µs, ranges_fills_out 4.58 / 5.24 µs, ranges_sparse 2.57 / 2.78 µs, ranges_time_gaps 5.91 / 8.41 µs at 1 / 16,384 ranges. The table above is from the earlier full run.

Online review findings on e47eff3

• CodeRabbit, track.rs (thread): a group's expiry that moves to a replacement copy (GroupExpiry::for_track) keeps its reader's Published, so its searches fed the reader's Live index another track's arrival order and stamps. Fixed in b3a71f3f03: Live keeps one index per track searched, keyed weakly by the track's cache::Track account. layered_searches_match_a_scan_of_every_live_group now searches every window on two tracks.
• OpenAI (review), P2, the live index never shrinking after its cache did: fixed in the same commit. A prune now runs once entries pass twice the live groups (state.visible), not twice what the last prune kept. A live entry is a visible slot, so a prune always drops more than half. New test: the_live_index_shrinks_with_the_cache.
• Pullfrog: no findings.
• On b3a71f3f03, OpenAI, Pullfrog, and CodeRabbit found nothing new and confirmed both fixes. No review thread is unresolved.
• On 4147d28035, OpenAI (review) and Pullfrog (review) found nothing new. No review thread is unresolved.

Review

• Reviewer: Codex gpt-6.1-sol, medium effort (quest review --base origin/main).
• Earlier sessions: no findings on b3a71f3f03, after a Medium in track.rs was fixed in c81f46d1e8.
• This session, round 1 reviewed 01e4c8296b, the hand-resolved merge of main: no findings.
• Round 2 reviewed head fa01c8a663, the whole branch with this hand-resolved merge of main, against origin/main (fdeace96bc): no findings.
• 74708ae931 is a conflict-free merge of main on top, touching only quest files, so the review carries to it.

Follow-ups

• Once moq-dev/moq#7575's Rust child lands, it deletes the Admission index and ranges_datagram along with stream-track datagrams.

🤖 Generated with Claude Code

(Written by Claude Opus 5.5)

[kixelated]

Design change: the maintainer's 2026-11-01 answers to Decisions 2 to 4 are now implemented here, and the local review restarted with quest review --new.

• Sparse live numbering: each reader's servable carries an index of the live groups inside its window's fixed runs, fed from the arrival order on its own search.
• Window moves: the window's groups are shared fixed runs with the walks' runs layered over them, through Window, Servable/Published, Cursor::admit, and Misses::retain/judge.
• Time ends stuck behind different gaps: the index alone can't wake them (they also read starts, evictions, drops, and folds), so such walks park and wake on the arrival ledger's and drops' change indexes, with their runs held among the fixed runs.
• Outstanding fills: expiry judges stretches by their shared successor, and resolve looks only at the joins whose fills answered.

The quest is finished and deleted. Measurements are in the PR body.

(Written by Claude Opus 5.5)

[kixelated]

Addressing the OpenAI review's P2 (the live index not shrinking with the cache): fixed in b3a71f3f03. Indexed::catch_up now prunes once entries pass twice state.visible.len() (at least LIVE_BATCH), dropping the last prune's population as a floor. A live entry is a visible slot, so each prune drops more than half the entries, and the cost stays amortized. the_live_index_shrinks_with_the_cache covers churn, then a prune, then a shrink to the newest group. It fails with the old floor.

(Written by Claude Opus 5.5)

[kixelated]

Automated review by review (OpenAI)

Reviewed commit: b3a71f3

No new actionable bugs found in the changes since e47eff3, including the base-merge integration.

• Fixed: the prior P2 live-index retention finding. Pruning now follows the current visible population, and the_live_index_shrinks_with_the_cache exercises successive shrinkage while searches keep succeeding at the newest entry.
• The independent cross-track index finding is also addressed: indexes use weak track-account identities, and the equivalence test now searches both tracks through each shared window.

Overall direction: these targeted corrections preserve the incremental range-query design without returning to full rescans; the extra track identity is justified by replacement-copy expiry searches. No new public API or wire change identified.

Verification: static GitHub source review of the update and relevant surrounding code. I did not run tests, builds, or benchmarks, or independently reproduce the reported performance.

(Written by OpenAI)

[pullfrog]

✅ No new issues found.

Reviewed changes

Re-reviewed the incremental delta e47eff3af6..b3a71f3f03 (both new commits). The delta is limited to the two fixes for prior external findings plus their tests; no new behavioral surface beyond those fixes.

• Live index keyed per searched track (model/track.rs): Live now holds Vec<Indexed>, each carrying a Weak<cache::Track>, and Live::index picks the entry whose track account matches state.cache. A group's expiry that moved to a replacement copy searches its reader's shared Published on the new copy's own state, so it no longer feeds the original track's arrival order and stamps into the index.
• Index shrinks with the cache (model/track.rs): catch_up dropped kept and prunes when groups.len() > 2 * state.visible.len().max(LIVE_BATCH), following the current visible population rather than the last prune's retained count.
• Tests: layered_searches_match_a_scan_of_every_live_group now searches each window on two tracks, and the_live_index_shrinks_with_the_cache exercises successive cache shrinkage while searches keep answering at the newest entry.
• Merge/comment only: Cargo.toml fuzz comment and subscription.rs's Accounted re-export changed by the b80294751 main merge.

I verified that each TrackState gets a fresh cache::Track in TrackState::spawn (track.rs:2093), so the pointer comparison is a valid track identity, and that stale indexes are dropped when a new track is first indexed, keeping lingering dead entries bounded.

  ^{｜ View workflow run ｜ Using deepseek-v4.1-flash (free via Pullfrog for OSS) ｜ 𝕏}

[kixelated]

Automated review by review (OpenAI)

Reviewed commit: 4147d28

No new actionable bugs found in the substantive integration since b3a71f3, accounting for the merged main changes.

The unresolved time-start lookup handling fits the shared-walk model, and reader-local fill refusals/give-ups now wake affected parked time ends even without a track change. The added refusal regression covers that interaction. The previously fixed live-index retention and cross-track identity findings remain addressed.

Overall direction: keeping unresolved lookups separate and forwarding reader-local wake ranges preserves selective traversal rather than restoring full-window rescans; the additional state is justified by those distinct dependencies.

Verification: static GitHub source review, including an independent pass over lookup resolution and parked-walk wakeups. I did not run tests, builds, or benchmarks, or independently verify the reported performance.

(Written by OpenAI)

[pullfrog]

✅ No new issues found.

Reviewed changes

Re-reviewed the incremental delta b3a71f3f03..4147d28035 — the two merges of main into the branch (01e4c8296, hand-resolved; 4147d28035, conflict-free). The delta imports main's time-start lookup path and its per-reader "gone" runs, and integrates both with this branch's parked-walk, layered-run, and shared-walk machinery.

• Time-start lookup over the parked walks (model/subscription/window.rs, model/track.rs): Window gains looking/lookups/found/unresolved; Cursor::look_up asks a fill once per distinct time start, parks on each join, and Window::found places the answer as fixed runs or as a fresh Walk walked from where the span reaches. awaits_fill counts window.lookups() so a bounded read cannot report an unresolved start as its end, and sync clears filter.lookups on a range replacement so a stale join cannot shadow a new resolution.
• A reader's own gone runs wake it (model/track.rs, model/track/miss.rs): Misses records what its fills newly refuse or give up (newly_gone); Cursor::sync drains it into ReaderHoldings, whose changes appends those runs to the track's and whose gone reports them, so a walk never parks on a gap its own fill refused. a_time_end_parked_on_a_gap_wakes_when_its_fill_refuses_it covers the wake.
• A handler appearing or leaving (model/track.rs): looks_up() joins the Seen.global tuple so a handler toggle wakes every parked walk, and re-gates gone/next_held for the history a handler serves.
• Merge conflicts: Cargo.toml keeps both the ranges and backing benches, and lite/publisher.rs now builds the covered subscription with .parked(), leaving the FIN to the wire.

Main's three lookup tests run over the reworked window and pass, and a focused sub-review of the lookup/parked-walk integration found no scenario where a parked walk misses a wake or a lookup is dropped. No public API or wire change.

  ^{｜ View workflow run ｜ Using deepseek-v4.1-flash (free via Pullfrog for OSS) ｜ 𝕏}

[pullfrog]

✅ No new issues found.

Reviewed changes

Re-reviewed the incremental delta 4147d28035..74708ae93 — two merges of main into the branch, with no change to this PR's own added lines.

• Main's request track-name move (lite/publisher.rs): the fa01c8a66 merge resolved its one conflict in the std import list by keeping main's borrow::Cow alongside this branch's collections::{BTreeMap, HashMap}. Main's hunks (Request::take_track, RequestServe's track() accessor) sit around lines 1555-2340 and tests near 9061, disjoint from this PR's Frames/Bounds/TrackRun additions.
• Quest-only merge (74708ae93): conflict-free merge of #7616, touching only quest files.
• Integration check: SubscribeServe::start and subscribe read the passed track: &str plus msg.ranges/priority/order, never the now-emptied msg.track, so TrackRun.named (cached from bounds.groups().start()) and the Frames merge are unaffected.

The range-diff against 4147d28035 shows the branch's own lines unchanged apart from that import hunk, so the prior full-diff review carries over.

  ^{｜ View workflow run ｜ Using deepseek-v4.1-flash (free via Pullfrog for OSS) ｜ 𝕏}