Skip to content

feat(net)!: add JavaScript socket endpoints and bound track requests by their lifetime - #7884

Merged
kixelated merged 30 commits into
mainfrom
quest/m1/socket/js
Nov 6, 2026
Merged

kixelated merged 30 commits into
mainfrom
quest/m1/socket/js

Conversation

@kixelated

@kixelated kixelated commented Nov 4, 2026 •

Copy link
Copy Markdown
Collaborator

Problem

JS lacks Rust's endpoint/session socket API. Its fixed ten-second track-answer budget also prevents applications from holding requests for acceptance.

Approach

  • Export socket.Endpoint, separate TCP/track accept queues, and incoming accept/reject handles. Announce one hidden broadcast per endpoint, with unlisted session tracks and a return-routability check before acceptance.
  • TCP uses one group each way with FIN/RST and second-group rejection. Track sessions expose existing producers and reliable ascending subscribers. Track sessions start at group 0 and skip unavailable messages; TCP joins its only group live.
  • Let all on-request track operations wait until demand ends, cancellation, refusal, or session closure. info({ signal }) provides caller cancellation.
  • Preserve main's newer cancellable stream-credit queues and lite-07 Range IDs. Cancellation releases queued opens without resetting their unused streams; returning credit serves the next request. Discovery follows its consumer lifetime, and IETF advertisements retain their answer budget.
  • Add Rust/JS relay lanes, lifecycle and controlled early-burst tests, docs, migration notes, and endpoint-count × sessions-per-endpoint benchmarks. Fix the Python README's missing “wait” and split the backpressure explanation into two sentences.

Impact

  • Additive @moq/net.socket: Endpoint, TcpStream, TrackStream, Incoming, and cancellation Options. JS methods use dialTcp/acceptTcp and dialTrack/acceptTrack. Streams expose existing producer/consumer handles plus peer and session.
  • Additive track.InfoOptions and Consumer.info(options?).
  • Breaking behavior: unanswered subscriptions, info lookups, fetches and fills no longer fail after ten seconds. Close the subscriber or abort its signal to bound the wait. Stream-credit waiting uses main's existing cancellation queue. See migration notes.
  • Terminal routed metadata refusals now reject info() with existing errors.
  • No new wire fields, framing, negotiation, Rust production API, or package versions. Every hop needs lite-07 (currently explicit moq-lite-07-wip) or IETF support. Endpoints fail as unsupported while their origin has a pre-07 lite client session.
  • JS TCP remains unpaced: a reader falling behind its retained group tail fails with eviction. JS fastest-reader pacing owns that work.

Decisions

  • ✅ Lifecycle-bound requests and cancellable stream-credit queues follow the maintainer decision and the later merged scheduler.
  • ✅ Follow the final Rust/Python decisions: track readers start at group 0 and skip refused, dropped, or evicted groups. TCP joins its single group live with SUBSCRIBE plus joining FETCH/fill. The merged JS subscriber rejects cut groups and reports them dropped, including after Largest at object 0; no duplicate adapter implementation is added.
  • ✅ Both TCP and track burst tests, plus Rust/JS socket interop, run over lite only as requested in this iteration. This intentionally narrows the sibling PR's explicit TCP exception. Generic IETF join/FETCH/fill tests retain their draft coverage.
  • ✅ Use main's unified Origin.Route API, retaining hidden setup only where a test must fill before announcement.

Alternatives

Reuse the settled Rust convention and group/track handles. The merged moq-dev/moq#7883 supplies the generic IETF fixes; this PR reuses them.

Validation

Pinned Nix shell, expensive commands through quest gate:

  • just ci origin/main: passed, exit 0 on 2471c51bd4. JS net: 2,998 passed, 3 skipped, 0 failed; selected Rust: 274 passed, 17 existing skips; Python: 125 passed. All selected build, lint, tooling, flake, and quest checks pass.
  • env -u MOQ_QUEST quest gate just test interop --all: passed, exit 0 on 5dea81f9a1. The subsequent main merge changes quest paths and comments only; socket code and tests are identical. All publishers/subscribers, auth, finished media, FETCH/history, late joins, tails, replacement/compression/claims/drop lanes, and Python/Rust sockets pass. All 8 Rust/JS socket pairs pass over lite-07, in both roles and protocols under Node and Bun.
  • Endpoint count × sessions per endpoint benchmark: all 36 cells completed successfully.
  • The initial focused 130-test run and the added Largest-at-object-0/2 regressions pass. Socket burst coverage is lite-only; the observed draft-18 scheduling failure is recorded below.

Review

Claude Code, claude-opus-5-5, medium effort, round 3 reviewed 2471c51bd4fcda45e51b9f6c738843f2ca988832 against origin/main (37f8305d43d8602de4dbe53808608a3d7d7f8a04): no findings.

Round 1's missing TCP group 0 being reported as clean FIN is fixed in 68bd23a088. Deterministic missing/dropped-group regressions and the valid empty-group FIN case pass. Rounds 2 and 3 report no findings. The final merge preserves completion of the JS socket quest while taking main's moved dependency paths; all 580 quest documents validate.

Follow-ups

  • Investigate the draft-18 early-answer joining-FETCH scheduling failure before adding future IETF burst coverage. The fixture passed alone but received [] after a single clock.advance(Nano.zero) in the full CI batch. It was immediately quarantined, then all IETF socket burst variants were removed to apply the requested lite-only scope. Missing TCP group 0 now fails explicitly, covered by deterministic regressions. Generic joining-FETCH tests stay enabled.
  • The existing reload close-drain investigation owns the earlier reproduced manual-clock failure. This PR keeps that case quarantined and records its failure evidence in the quest.
  • The first unscoped CI run also hit the existing CARLA loader-environment regression under Codex's BASH_ENV hook: nix develop --command bash -c 'bash sim/python.test.sh' from demo/carla fails on this branch and unchanged 5480d5109f59f52f1b98668fff5a90be454060f4 after initializing each checkout's shell snapshot. The hook restores PYTHONPATH/compiler search paths inside the fake compiler fixture. An uninitialized base checkout passes. No open fix was found; investigate the shell snapshot's repeated environment restoration separately.

(Written by GPT-6.1 Sol with xhigh reasoning effort)

Quest-Claim: 67eba2eda1b1be82
kixelated and others added 8 commits November 4, 2026 08:37
Co-Authored-By: GPT-6.1 Sol <noreply@openai.com>
Co-Authored-By: GPT-6.1 Sol <noreply@openai.com>
Co-Authored-By: GPT-6.1 Sol <noreply@openai.com>
Co-Authored-By: GPT-6.1 Sol <noreply@openai.com>
Co-Authored-By: GPT-6.1 Sol <noreply@openai.com>
Co-Authored-By: GPT-6.1 Sol <noreply@openai.com>
Fix socket interop planning and refuse known late session queries before the fresh refusal probe.

Co-Authored-By: GPT-6.1 Sol <noreply@openai.com>
kixelated and others added 4 commits November 4, 2026 09:51
Co-Authored-By: GPT-6.1 Sol <noreply@openai.com>
Keep refusal independent of data-session queries and preserve the original single rejection assertions in both interop clients.

Co-Authored-By: GPT-6.1 Sol <noreply@openai.com>
@kixelated kixelated changed the title JS sockets feat(net): add JavaScript socket endpoints Nov 4, 2026
@kixelated

Copy link
Copy Markdown
Collaborator Author

Maintainer decisions (2026-11-04, /quest-unblock):

  1. Request-setup budget: ✅ demand-driven, like Rust. Replace fixed application-answer budgets for all on-request tracks with demand, cancellation, fail-fast stream credit and session closure, and remove the .socket/<id> namespace exception. Not chosen: transport-only setup budgets with an internal phase split.
  2. Generic IETF default (shared with feat(ffi)!: expose socket endpoints and preserve IETF starts #7883): as an IETF publisher, an incoming filterless SUBSCRIBE starts at the latest group (live join). As an IETF subscriber, when our start is Some(0) and max-age is None, send SUBSCRIBE plus a FILL/FETCH to start at zero. Not chosen: changing filterless SUBSCRIBE to replay from group 0.

Record both in the quest Plan and remove the items.

(Written by Claude Opus 5.5)

kixelated and others added 4 commits November 4, 2026 14:09
A subscribe, track info, fetch, or fill now waits for its answer until its
demand ends, its caller cancels, stream credit fails fast, or the session
closes, as in Rust, replacing the ten-second answer budget and the
`.socket/<id>` namespace exemption. Lite's first subscribe setup watches
demand like its reopen and IETF's, IETF's history setup ends with the
info lookup, and `track.Consumer.info()` takes an optional `signal`.

Socket dials and accepts fail as unsupported while the endpoint's origin has
a moq-lite client session before 07, matching Rust (#7888).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@kixelated kixelated changed the title feat(net): add JavaScript socket endpoints feat(net)!: add JavaScript socket endpoints and bound track requests by their lifetime Nov 4, 2026

Copy link
Copy Markdown
Collaborator Author

Design change: Apply the maintainer's lifecycle-bound track-request decision together with main's later stream-credit scheduler. Track answers have no fixed timer; pending stream opens wait in the cancellable priority queue. Cancellation gives unused credit to the next open. Discovery keeps its consumer lifetime, and IETF advertisements retain their answer budget. This replaces the old namespace exemption and fail-fast-credit assumptions. The generic IETF absolute-start correction remains owned by moq-dev/moq#7883.

(Written by GPT-6.1 Sol with xhigh reasoning effort)

kixelated and others added 2 commits November 4, 2026 22:27
Co-Authored-By: Codex <noreply@openai.com>
kixelated and others added 7 commits November 5, 2026 16:48
…emantics

Use Route handles, join TCP halves live, and exercise lite-only track bursts. Preserve the merged IETF whole-group skip behavior and document missing-message delivery.

Co-Authored-By: GPT-6.1 Sol <noreply@openai.com>
Keep the assertions for investigation and retain draft-20/22 TCP burst coverage. Simplify the lite-only socket interop loop for shellcheck.

Co-Authored-By: GPT-6.1 Sol <noreply@openai.com>
Co-Authored-By: GPT-6.1 Sol <noreply@openai.com>
Co-Authored-By: GPT-6.1 Sol <noreply@openai.com>
Co-Authored-By: GPT-6.1 Sol <noreply@openai.com>
…points-7884

# Conflicts:
#	quest/m1/socket/js.md
#	quest/m3/stream/js.md
@kixelated
kixelated marked this pull request as ready for review November 6, 2026 01:37
@pullfrog

pullfrog Bot commented Nov 6, 2026 •

Copy link
Copy Markdown
Contributor

This run was cancelled before it finished.

Pullfrog  | View workflow run | via Pullfrog

@kixelated
kixelated added this pull request to the merge queue Nov 6, 2026

@kixelated kixelated left a comment

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Automated review by review (OpenAI)

Reviewed head 2471c51bd4fcda45e51b9f6c738843f2ca988832 against 37f8305d43d8602de4dbe53808608a3d7d7f8a04.

No actionable findings from this static review. I checked the endpoint handshake and return-routability checks, cancellation and session teardown, TCP FIN/RST and second-group handling, lifecycle-bound track requests, metadata refusal propagation, and the added regression/interop coverage.

The earlier missing-group-as-FIN issue described in the PR is addressed: a missing or dropped TCP group 0 now fails, while an explicitly empty group still ends cleanly. The implementation follows the settled socket convention and cancellable stream-credit model; no additional code change is requested by this review.

Verification limits: GitHub source/diff review only; I did not execute tests or independently reproduce the reported local validation. At posting, Quest passed and the main Check job was still running. IETF early-burst scheduling remains the explicitly documented follow-up outside this iteration's lite-only burst/interop coverage.

Merged via the queue into main with commit b891adb Nov 6, 2026
5 of 8 checks passed
@kixelated
kixelated deleted the quest/m1/socket/js branch November 6, 2026 01:45
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant