Skip to content

feat(net): fetch past group ranges over IETF in JavaScript - #7793

Merged
moq-bot[bot] merged 15 commits into
mainfrom
quest/m1/interop-coverage/js-ietf-fetch
Nov 4, 2026
Merged

moq-bot[bot] merged 15 commits into
mainfrom
quest/m1/interop-coverage/js-ietf-fetch

Conversation

@kixelated

@kixelated kixelated commented Nov 4, 2026 •

Copy link
Copy Markdown
Collaborator

Problem

JavaScript could recover the head of a current IETF group but could not fetch past groups. fetchGroup() was disabled, and past-range cache misses had no FETCH handler.

Approach

  • Resolve bounded history and metadata-only reads with TRACK_STATUS, then map missing runs to standalone FETCH on draft 18 or filtered FETCH on drafts 20/22.
  • Reuse the joining FETCH object decoder and streamed payload receiver. Preserve joining-head stitching, timestamps, sparse group deltas, cached history, and cancellation ownership. A failed live widening fails its live readers while history readers keep their valid groups.
  • Cross Rust, Node, and Bun as FETCH publishers and subscribers. Add Node/Bun subscribers to the existing on-demand history cells. Test multi-group, overlapping, and omitted-start ranges directly against pinned Moxygen, checking every payload byte.
  • Complete the quest and its final-child interop coverage epic. Preserve references to completed decisions as historical links.

Impact

  • Public API: no shape changes or migration. Existing track.Consumer.fetchGroup() and past group ranges now work over moq-transport drafts 18, 20, and 22.
  • Wire: no format or version changes. Uses existing TRACK_STATUS, FETCH, FETCH_OK, and fetch streams. First-party publishers still refuse multi-group FETCH; those ranges need a publisher that supports them, as the settled plan specifies.
  • Choices: request ascending wire delivery and retain the caller's local read order. Unsupported subgroups, datagrams, and object gaps inside a group fail the FETCH. Live streams still owed by the existing arrival ledger are not fetched again as cache misses.

Alternatives

SUBSCRIBE cannot retrieve a finished track's past groups. A separate public fetch API would duplicate the range and fill model introduced by JS ranges.

Validation

Head cbe9bb44b9c8ceb0ccc85a6977c729abe740ef15 merges origin/main at e6be13d5f8 and fixes both review findings. It passed locally through quest gate:

  • js/net bun run check and bun test: 2867 pass, 0 fail. Biome is clean on the changed files.
  • just test interop --tail --all --versions ietf: 88 PASS on moq-transport drafts 18, 20, and 22. This covers every cached and on-demand FETCH pairing, late join, tails, status, and GOAWAY.
  • just test ietf-conformance --fetch: Moxygen sparse groups 2,4,6 and overlapping groups 0,2,4, 18 frames per runtime.

Earlier head 15b8eaa284 also passed just ci and just test interop --all. The merge conflicted only in test/interop/clients/js-native/tail.ts. The resolution keeps this PR's fetch mode and drops the datagram mode that main removed. Among today's js/net changes, main brought only moq-dev/moq#7721, which touches the publisher and payload paths, not the IETF subscriber this PR changes.

Review

mechanical fix by codex / Codex: Merged base and re·solved all three conflicts preserving both FETCH coverage changes; passed git merge --no-edit base; bun run --cwd js/net build, check, test (2882 pass); bun run --cwd test/interop/clients/js-native check; bun x tsc --project test/interop/clients/js-native/tsconfig.json --noEmit false --rootDir . --outDir /tmp/7793-native-build; bash -n and shellcheck -x test/interop/interop.sh test/ietf-conformance/run.sh; bash test/interop/relay-log.test.sh; python3 test/interop/staging.test.p (head 0987bf9)

Codex, gpt-6.1-sol, medium effort, local quest review round 1 reviewed head cbe9bb44b9c8ceb0ccc85a6977c729abe740ef15 and reported no findings.

The online reviews of df120cee raised two findings, both fixed in cbe9bb44b9:

  • Pullfrog thread and OpenAI P1: the history wake shared the idle symbol, so narrowing a widened read back to history spun the serve loop. It now has its own terminal, which closes the live SUBSCRIBE.
  • OpenAI P2: a relative end on a live track with no reported Largest never subscribed, so it never resolved. #needsLive now opens SUBSCRIBE until a group is known.

js/net/src/ietf/range.test.ts covers both on every draft, and each test fails without its fix.

Earlier rounds (Claude Code, through df120cee) fixed these: failed live widening preserves history; settled joins no longer block refetch after eviction; reported Largest only extends fill bounds for an IETF copy without a live feed; omitted starts keep their oldest-group meaning; and a trailing NonExistent marker preserves valid group payloads. Those rounds refuted automatic chaining of short FETCH answers. The settled plan keeps first-party multi-group requests refused, as the publisher does. Uncovered groups after a short FETCH_OK fail, as Rust's subscriber does.

Follow-ups

None proposed. Publisher on-demand IETF FETCH dispatch remains a separate task. Its integration should preserve run_fetch_round(publisher, subscriber, mode) and the Rust/Node/Bun subscriber loops when enabling the IETF history cells; those cells remain skipped pending that publisher work. No publisher dispatcher or object encoder changes are included here.

No maintainer decisions remain.

(Written by GPT-6.1 Sol; updated by Claude Opus 5.5)

kixelated and others added 12 commits November 3, 2026 18:17
Co-Authored-By: GPT-6.1 Sol <noreply@openai.com>
Co-Authored-By: GPT-6.1 Sol <noreply@openai.com>
Co-Authored-By: GPT-6.1 Sol <noreply@openai.com>
Co-Authored-By: GPT-6.1 Sol <noreply@openai.com>
Co-Authored-By: GPT-6.1 Sol <noreply@openai.com>
… markers

Co-Authored-By: GPT-6.1 Sol <noreply@openai.com>
Co-Authored-By: GPT-6.1 Sol <noreply@openai.com>
Keep upstream quest changes intact after the base merge and adapt FETCH fixtures to explicit publisher priorities.

Co-Authored-By: GPT-6.1 Sol <noreply@openai.com>
Co-Authored-By: GPT-6.1 Sol <noreply@openai.com>
Co-Authored-By: GPT-6.1 Sol <noreply@openai.com>
Co-Authored-By: GPT-6.1 Sol <noreply@openai.com>
@kixelated kixelated changed the title JS subscribers FETCH over moq-transport feat(net): fetch past group ranges over IETF in JavaScript Nov 4, 2026
@kixelated
kixelated marked this pull request as ready for review November 4, 2026 03:59
@coderabbitai

coderabbitai Bot commented Nov 4, 2026 •

Copy link
Copy Markdown
Contributor

Warning

Review limit reached

You've used all free OSS reviews for now. Wait for the free limit to reset to keep reviewing this public repository.

Next included review available in 3 minutes.

Check out review usage here.

View limit details

Limit details: You’ve used all 4 included reviews currently available.

Learn how review limits work.

Review configuration:

Run configuration
  • Source mode: Auto
  • Repository: Unavailable
  • Branch: Unavailable
  • File: No configuration file from this repository applied
  • Commit: Unavailable
  • Layers: Organization UI
  • Review profile: CHILL
  • Plan: Advanced
  • Run ID: 7222ced5-2b52-44d1-bb66-e0f4ad8f629e
Commits

Reviewing files that changed from the base of the PR and between e6be13d and cbe9bb4.

Files selected for processing: 26
  • js/net/MIGRATE.md
  • js/net/src/fill.ts
  • js/net/src/goaway-requests.test.ts
  • js/net/src/ietf/join.test.ts
  • js/net/src/ietf/range.test.ts
  • js/net/src/ietf/range.ts
  • js/net/src/ietf/subscriber.test.ts
  • js/net/src/ietf/subscriber.ts
  • js/net/src/ietf/tail.test.ts
  • js/net/src/integration.test.ts
  • js/net/src/internal.ts
  • js/net/src/track.ts
  • quest/m1/README.md
  • quest/m1/ietf-drop/README.md
  • quest/m1/interop-coverage/README.md
  • quest/m1/interop-coverage/js-ietf-fetch.md
  • quest/m2/audit-interop.md
  • quest/m2/ietf-hole-marker.md
  • quest/m2/publish-done-codes.md
  • test/ietf-conformance/README.md
  • test/ietf-conformance/run.sh
  • test/interop/README.md
  • test/interop/clients/js-native/ranges.ts
  • test/interop/clients/js-native/tail.ts
  • test/interop/interop.sh
  • test/justfile
  • Autofix · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

Share

@kixelated kixelated left a comment

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Automated review by review (OpenAI)

Reviewed head df120ce.

  • P1: Stop the live subscription when demand narrows back to history (js/net/src/ietf/subscriber.ts:889–892, 930–937). Start a bounded history reader, widen it to live and accept SUBSCRIBE, then close the live reader while retaining history. #historyOnly resolves as idle, but history still makes demand.wanted defined; the loop continues and races that already-resolved promise forever. This spins the JavaScript microtask queue and prevents the session and page from making progress. Distinguish the history-only wake from an unwanted-demand wake and recheck whether live data is still needed before closing. Test a successful widening followed by narrowing.
  • P2: Subscribe when an empty track must resolve a relative end (js/net/src/ietf/subscriber.ts:986–990, 1078–1081). For a live, initially empty track, a bounded range from absolute group 0 through a relative end has no Largest in TRACK_STATUS. bounded is true but wireSpan.end is undefined, so #needsLive is false; there is no live SUBSCRIBE or group eligible for FETCH. The relative endpoint waits for the first group, but it never arrives even when the publisher writes it. Open the live feed when the relative end cannot yet be resolved, and cover this empty-track case across the supported drafts.

Verification: GitHub-only static inspection of the subscription lifecycle, range model and tests. I did not run tests or interop sessions.

(Written by review (OpenAI))

@pullfrog pullfrog Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Important

One finding before merge: a resolved history terminal can busy-loop the subscription's serve loop.

Reviewed changes

  • Past-range resolution: bounded and metadata-only reads go through TRACK_STATUS and then standalone (18/20) or filtered (22) FETCH, with live SUBSCRIBE only for ranges that reach past the reported edge.
  • Range receiver: the new RangeFetch decodes the fetch stream, writes groups into the fill, stitches against live data, handles EndOfRange, and enforces FETCH_OK's End Location.
  • Live-widening lifecycle: failures fail live readers beyond the edge while history readers and one-shot fetches keep their completed groups; cacheTrack/refuseLive/subscribing carry that split in track.ts.
  • Fill tracking: wantedRuns/fillChanged exports, owedRuns/fillEdge, and a fetch sink flag so one-shot fetches do not count as live subscribers.
  • Tests and interop: per-draft range tests, a Rust/Node/Bun FETCH matrix, Moxygen multi-group conformance, and completed quest docs.

Pullfrog  | Fix all ➔ | Fix 👍s ➔ | View workflow run | Using deepseek-v4.1-flash-1017 (free via Pullfrog for OSS) | 𝕏

Comment thread js/net/src/ietf/subscriber.ts Resolved
kixelated and others added 2 commits November 3, 2026 22:38
Resolve tail.ts: keep the fetch mode and drop the datagram mode main removed.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…k's relative end

The history wake shared the idle symbol, so a widened read narrowed back to history
re-raced its settled promise forever. Give it its own terminal so the live SUBSCRIBE
closes while history readers keep reading.

A relative end on a track with no Largest resolves at the first group, which only the
live feed delivers, so open SUBSCRIBE for it.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@kixelated

Copy link
Copy Markdown
Collaborator Author

Responding to the automated OpenAI review of df120cee. Both findings are fixed in cbe9bb4:

  • P1, busy loop after narrowing back to history: the history wake has its own terminal now, so narrowing ends the live SUBSCRIBE instead of re-racing a settled promise.
  • P2, relative end on an empty track: #needsLive opens SUBSCRIBE when a range ends relative to Largest and TRACK_STATUS reported none. The end then resolves at the first live group.

New tests in js/net/src/ietf/range.test.ts cover both cases on drafts 18, 20, and 22. Each fails without the fix.

(Written by Claude Opus 5.5)

@kixelated
kixelated added this pull request to the merge queue Nov 4, 2026

@kixelated kixelated left a comment

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Automated review by review (OpenAI)

Reviewed head cbe9bb4, following df120ce.

Both previously reported findings are addressed. The history-only wake now ends the widened live subscription instead of spinning on a settled promise, while the history producer remains available. An unresolved relative end on an initially empty track now opens a live subscription for its first group. The added tests cover both paths on drafts 18, 20, and 22. No additional actionable issue found in this delta.

Verification limits: GitHub-only static review of the subscriber lifecycle, surrounding history loop, and tests, including an independent pass. No tests or interoperability sessions were run.

(Written by review (OpenAI))

@pullfrog pullfrog Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

✅ No new issues found.

This run re-reviewed the two commits since df120cee, which fix the previous live-widening busy-loop, subscribe for an empty track's relative end, and resolve the main merge in the js-native tail client.

Reviewed changes

  • History-only widening terminal: #historyOnly now resolves to its own historyEnded symbol, so narrowing a widened read back to history breaks the serve loop and closes the live SUBSCRIBE instead of spinning on the settled idle wake. The prior Pullfrog finding is addressed, with a per-draft test that fails without the fix.
  • Empty-track relative end: #needsLive now subscribes when a bounded range has no absolute wire end and no Largest is known yet, so a relative end on an initially empty track resolves once the first group arrives. Covered per draft.
  • Merge resolution: the js-native tail interop client drops the datagram mode removed on main while keeping fetch; no stale references remain.

Pullfrog  | View workflow run | Using deepseek-v4.1-flash-1017 (free via Pullfrog for OSS) | 𝕏

@github-merge-queue
github-merge-queue Bot removed this pull request from the merge queue due to a conflict with the base branch Nov 4, 2026
@moq-bot

moq-bot Bot commented Nov 4, 2026

Copy link
Copy Markdown
Contributor

quest land watch: the merge queue removed this pull request at cbe9bb4 because it conflicts with main (merge_conflict).

The land fixer works on it in the background, and the watcher re-enqueues it at the back of the queue once it is fixed or judged flaky, or hands it back here with what it needs.

(written by quest land watch)

@moq-bot
moq-bot Bot enabled auto-merge November 4, 2026 08:08
@moq-bot

moq-bot Bot commented Nov 4, 2026

Copy link
Copy Markdown
Contributor

codex / Codex land fixer note:

Resolved the three merge conflicts by keeping both migration entries and combining the PR’s subscriber matrix with base’s IETF history FETCH coverage. Networking build and all 2,882 unit tests passed, as did native-client compilation, shell validation, and interop harness regression tests. Full live relay and Moxygen conformance runs were not performed.

(written by codex / Codex)

@moq-bot

moq-bot Bot commented Nov 4, 2026

Copy link
Copy Markdown
Contributor

quest land watch: armed auto-merge on this pull request at 0987bf9, so it joins the back of the merge queue once its required checks pass, with the land fixer's fix (codex / Codex): Merged base and re·solved all three conflicts preserving both FETCH coverage changes; passed git merge --no-edit base; bun run --cwd js/net build, check, test (2882 pass); bun run --cwd test/interop/clients/js-native check; bun x tsc --project test/interop/clients/js-native/tsconfig.json --noEmit false --rootDir . --outDir /tmp/7793-native-build; bash -n and shellcheck -x test/interop/interop.sh test/ietf-conformance/run.sh; bash test/interop/relay-log.test.sh; python3 test/interop/staging.test.p.

(written by quest land watch)

@moq-bot
moq-bot Bot added this pull request to the merge queue Nov 4, 2026

@kixelated kixelated left a comment

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Automated review by review (OpenAI)

Reviewed head 0987bf9, following cbe9bb4 and the base merge. No actionable issue found in the merge integration. The cross-runtime history FETCH matrix now composes with main's on-demand IETF publisher support: the planned and executed cells both include Rust, Node and Bun subscribers, and the earlier subscriber fixes remain intact.

Verification limits: GitHub-only static inspection of the merge, interop planning and execution paths. I did not run the interop matrix or validate its runtime results on this head.

(Written by review (OpenAI))

@pullfrog pullfrog Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

✅ No new issues found.

Reviewed changes

main was merged in at dcc730b, pulling #7794's on-demand JavaScript IETF FETCH responder and reconciling the interop history cells with this branch's multi-subscriber harness.

  • Enabled the CLI FETCH history cells on every supported IETF draft now that the publisher side landed on main, crossing all Rust/Node/Bun subscriber pairs.
  • Updated the interop README and js/net/MIGRATE.md ordering, keeping both Unreleased entries and main's #7702 entries.
  • Confirmed tail.ts no longer references the removed datagram mode.

ℹ️ The PR body still says the IETF history cells are skipped

The ## Follow-ups note says publisher on-demand IETF FETCH is a separate task and that "those cells remain skipped pending that publisher work." After merging main, the cells now run on every draft, so that note and the matching cbe9bb44b validation claim are stale.

Technical details
# Stale follow-up note

## Affected sites
- PR #7793 body, `## Follow-ups` — says publisher on-demand IETF FETCH is separate and the IETF history cells remain skipped.
- PR #7793 body, `## Validation` — describes the `cbe9bb44b` interop run, which predates enabling the IETF history cells.

## Required outcome
- The description reflects that `#7794` landed and that this merge enabled the history cells on drafts 18, 20, and 22 for all subscriber runtimes.

Pullfrog  | View workflow run | Using deepseek-v4.1-flash-1017 (free via Pullfrog for OSS) | 𝕏

Merged via the queue into main with commit 10b355e Nov 4, 2026
7 of 10 checks passed
@moq-bot
moq-bot Bot deleted the quest/m1/interop-coverage/js-ietf-fetch branch November 4, 2026 08:16
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant