Skip to content

feat(relay)!: enforce bitrate caps on QUIC sessions - #7535

Merged
moq-bot[bot] merged 9 commits into
mainfrom
quest/m2/rate-quic
Nov 2, 2026
Merged

moq-bot[bot] merged 9 commits into
mainfrom
quest/m2/rate-quic

Conversation

@kixelated

@kixelated kixelated commented Nov 1, 2026 •

Copy link
Copy Markdown
Collaborator

Finishes quest/m2/rate-quic.md: the auth and relay half, on top of the stack half from moq-dev/moq#7094. A relay session over QUIC (tokio and io_uring) is now held to its token's publish_rate and subscribe_rate, and a capped session over QUIC is admitted instead of refused.

Summary

  • moq-auth. Rate (positive bits per second, at most 2^53 - 1) and additive publish_rate / subscribe_rate on Claims and Grant, as the maintainer decided on moq-dev/moq#7094. A capped role is written {"path": [...], "rate": bps}, never as legacy put/get, so released verifiers refuse it rather than drop the cap (tests/released_token.rs). Key scopes still refuse caps. moq auth serve copies a token's caps into its grant. Shared vectors in rs/moq-auth/rate.json.
  • js/auth mirrors it: publish_rate / subscribe_rate on parsed claims and grants, the object role on the wire, the same vectors.
  • Relay enforcement (rs/moq-relay/src/auth/caps.rs). A Pacer holds the QUIC connection to a session's Caps: the tokio path calls set_limits on the moq-tokio connection; the io_uring path hands each change to a task on the connection's worker through a kio channel. A session without a pacer (WebSocket, iroh, HTTP /fetch and /announced, moq --listen) refuses a capped grant at connect, in band, and at revalidation. publish_rate becomes the receive cap and subscribe_rate the send cap, bits to bytes rounded up.
  • Unions. In-band credentials join the caps: each direction takes the highest cap, and a grant giving that direction paths without a cap lifts it. Every change, from revalidation or a credential joining or leaving, reaches the live connection.
  • Pre-auth window. A server QUIC connection starts with only moq_net::transport::SETUP_RECEIVE_WINDOW (SETUP plus one AUTH, about 128 KiB), so a client cannot bank the full window before a cap applies. moq-tokio's Request::ok opens [quic] receive_window after the limits set through the new Request::with_quic_limits (the caps, or a peer's wider limits); the io_uring relay does the same before its SETUP reply.
  • Docs: doc/bin/relay/auth.md gains a Bitrate caps section; doc/lib/{rs,js} updated. The quest file is deleted and the prose that linked it now names the landed types.

Impact

  • Wire (auth JSON, not moq-lite): claims and grants accept the capped object role. Uncapped documents encode exactly as before.
  • moq-auth: additive Rate, InvalidRate, Claims::{publish_rate, subscribe_rate, with_publish_rate, with_subscribe_rate}, Grant::{publish_rate, subscribe_rate} (both structs are #[non_exhaustive]).
  • moq-net: new transport::SETUP_RECEIVE_WINDOW.
  • moq-tokio: new server::Request::with_quic_limits. Behavior: every server QUIC connection holds the SETUP window until Request::ok. A caller that used Request::quic().set_limits(..) before ok should use with_quic_limits instead, or ok reopens the configured window over it.
  • moq-relay: auth::Token gains publish_rate / subscribe_rate; Auth::admit and Lease::new refuse a capped grant (Forbidden, "this transport cannot enforce bitrate caps"); a viewer now receives a MAX_DATA once admitted.
  • js/auth (breaking): GrantSchema is now a zod pipe, so GrantSchema.shape is gone; parse with GrantSchema.parse. New encodeGrant(grant) writes a grant's caps into its roles.

Decisions

The maintainer accepted all five implemented recommendations on 2026-11-01 (in the /quest-complete run). The quest file is deleted by this PR, so this section is the paper trail.

  1. How moq-tokio holds and opens the pre-auth window.
    • (a) ✅ 2026-11-01: every moq-tokio server QUIC connection starts at SETUP_RECEIVE_WINDOW; Request::ok opens [quic] receive_window, after the limits a caller set with Request::with_quic_limits.
    • (b) Shrink only when the server opts in through a programmatic server::Config field.
    • (c) No new method: ok opens the window unless a set_limits already set one, tracked in hidden driver state.
  2. The pre-auth window's size and home.
    • (a) ✅ 2026-11-01: moq_net::transport::SETUP_RECEIVE_WINDOW, the largest SETUP plus one lite AUTH with framing (131,102 bytes), beside MIN_RECEIVE_WINDOW.
    • (b) SETUP only (about 64 KiB).
    • (c) A rounder margin such as 1 MiB.
  3. Which grants count in a union's caps.
    • (a) ✅ 2026-11-01: every live credential, request tokens included; a grant that gives a direction neither paths nor a cap has no say in it.
    • (b) Every grant counts in both directions.
    • (c) Only the connection credential and AUTH tokens count.
  4. The object role's strictness.
    • (a) ✅ 2026-11-01: exactly a path list and a rate; a missing rate or a bare-string path is refused.
    • (b) Accept an optional rate and a string path, as the reverted moq-dev/moq#5697 did.
  5. js/auth additions the mirror needed.
    • (a) ✅ 2026-11-01: export encodeGrant(grant), and let Key.sign / KeySet.sign accept publish_rate / subscribe_rate.
    • (b) No helper; document that an auth server writes the object role itself.

Choices made without asking

  • Rate::from_bps / as_bps / MAX and Display, mirroring the reverted API; bits to bytes round up, so 1 bps is 1 B/s.
  • Auth::admit stays public and refuses caps; the relay's QUIC paths use a crate-private admit_paced. The refusal is Forbidden (HTTP 403, session Unauthorized), recorded like any other.
  • The caps the connection holds are applied with its first limits before SETUP OK, so a capped window opens paced.
  • serve_listening spawns each Connection from its own function (spawn_connection), so the session future stays out of the accept loop's frame. With io-uring, the gate's debug build overflowed stack::debug_relay_load_and_run_fit_windows_stack: main needs 992 to 1024 KiB, and this branch tipped it past 1 MiB. The change brings it back under, at about main's margin.
  • Merging main's dot-segment refusal (moq-dev/moq#7366): the check moved into admit_paced, which every relay admission goes through, paced or not.
  • The cap tests run on real sockets and assert lower bounds on transfer time (about 1 s each, two per runtime), since the QUIC sim is not merged yet.

Tests

  • rs/moq-relay/tests/integration/bitrate_caps.rs, on the tokio listener and the io_uring workers: a capped publisher and a capped subscriber each take at least 0.7 s for 2 MiB at 8 Mbps (milliseconds uncapped); a client is blocked at the SETUP window before admission and credited once admitted.
  • auth::caps: unions take the highest cap and an uncapped grant lifts it; bits to bytes.
  • auth::refresh: in-band unions pace the live connection; a capped in-band token is refused without a pacer and the session keeps its grant; a revalidation into a cap ends an unpaced session.
  • auth: capped grants are refused on every transport without a pacer and admitted with one; a revalidated cap reaches the connection.
  • moq-auth: shared vectors, signing, scope refusal, released-verifier refusal. js/auth mirrors them.
  • peer_limits now expects a viewer's window to open after admission.
  • just check origin/main passes on 07316c1cdc363906a5fa3bd8749639ccea84a0bd (9637 Rust tests, including the io_uring ones).

Follow-ups

  • moq auth sign has no flags for caps, so a capped token can only be minted through the library.
  • Move the wall-clock cap tests onto the moq-quic sim once quest/m2/quic-sim.md lands.
  • The relay's debug startup still sits within 32 KiB of a 1 MiB stack on main too. Relay::run_inner's poll frame alone is about 431 KB. quest/m0/test-flakes-2/relay-stack-all-features.md could take this on as well: shrinking that frame fixes both the all-features overflow and this margin.
  • wire_stats::uring_*, peer_limits::uring_*, and bitrate_caps::uring_* sit outside the io-uring nextest group, so parallel runs can exhaust RLIMIT_MEMLOCK.

Review

mechanical fix by codex / Codex: Merged base and retained both QUIC Server initializer fields; passed cargo nextest run --locked -p path+file:///work/rs/moq-auth -p path+file:///work/rs/moq-net -p path+file:///work/rs/moq-tokio -p path+file:///work/rs/moq-relay --features moq-relay/io-uring; bun run --cwd js/auth check; bun run --cwd js/auth test; bun run --cwd js/auth build; rustfmt --check --edition 2024 rs/moq-tokio/src/quic/backend.rs; git diff --cached --check. (head b9fb280)

  • Reviewer: Codex (quest review), gpt-6.1-sol, medium effort, a new session for this iterate's branch.
  • Round 1 on 7397309693f66c62c442844109af48113120481e (the decided answers, the merge of main, and the stack fix): no findings.
  • Round 2 on 07316c1cdc363906a5fa3bd8749639ccea84a0bd (a second merge of main, with the auth.rs conflict resolved by hand): no findings.
  • Round 3 on the same head, --deep (gpt-6-astra, medium effort), resuming that session: no findings.
  • A fresh --deep --new session (gpt-6-astra, medium effort), round 1 on 07316c1cdc363906a5fa3bd8749639ccea84a0bd, so the security read is not anchored on the earlier rounds: no findings.
  • Not skipped under the simple-review rule: the change touches auth, caps, and public API.
  • No online-reviewer threads were posted while the PR was a draft.

(Written by Claude Opus 5.5)

🤖 Generated with Claude Code

kixelated and others added 5 commits November 1, 2026 01:28
Quest-Claim: 8f1e29551735ff28
Token claims and grants carry optional publish_rate and subscribe_rate caps
(moq_auth::Rate) as the {"path","rate"} object role. The relay holds QUIC
and WebTransport sessions on the tokio and io_uring listeners to them,
taking the union's highest cap, applying re-checked caps live, and refusing
caps on every transport that cannot enforce them. Every server QUIC
connection starts with only the SETUP window, opened once admitted.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…56111401a74

# Conflicts:
#	quest/m2/rate-websocket.md
#	rs/moq-tokio/src/server.rs
@kixelated kixelated changed the title QUIC enforces bitrate caps feat(relay)!: enforce bitrate caps on QUIC sessions Nov 1, 2026
kixelated and others added 3 commits November 1, 2026 14:04
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
With the io-uring feature the gate's debug build overflowed the 1 MiB stack in stack::debug_relay_load_and_run_fit_windows_stack: main sat within 64 KiB of the limit and this branch's admission changes tipped it over. Spawning each Connection from its own function keeps the session future, built on the stack before tokio::spawn moves it, out of serve_listening's frame, which every accept stacks on.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Resolves rs/moq-relay/src/auth.rs: the dot-segment refusal moves into admit_paced, which every relay admission, paced or not, goes through.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@kixelated
kixelated marked this pull request as ready for review November 1, 2026 22:41
@coderabbitai

coderabbitai Bot commented Nov 1, 2026 •

Copy link
Copy Markdown
Contributor

Review in Change Stack →

No actionable comments were generated in the recent review.

Recent review info
Run configuration
  • Source mode: Auto
  • Repository: Unavailable
  • Branch: Unavailable
  • File: No configuration file from this repository applied
  • Commit: Unavailable
  • Layers: Organization UI
  • Review profile: CHILL
  • Plan: Advanced
  • Run ID: 6f874e87-c98a-46e0-92b5-abbba5744a80
Commits

Reviewing files that changed from the base of the PR and between af1a0ce and b9fb280.

Files selected for processing: 49
  • doc/bin/relay/auth.md
  • doc/lib/js/auth.md
  • doc/lib/rs/moq-auth.md
  • js/auth/README.md
  • js/auth/api/index.api.md
  • js/auth/src/claims.test.ts
  • js/auth/src/claims.ts
  • js/auth/src/contract.test.ts
  • js/auth/src/contract.ts
  • js/auth/src/interop.test.ts
  • js/auth/src/key.test.ts
  • js/auth/src/key.ts
  • js/auth/src/set.ts
  • js/auth/src/wire.ts
  • quest/m2/README.md
  • quest/m2/quic-sim.md
  • quest/m2/rate-grant.md
  • quest/m2/rate-quic.md
  • quest/m2/rate-websocket.md
  • quest/m2/rtc-pacing.md
  • quest/m2/srt-rate-handle.md
  • rs/moq-auth/CHANGELOG.md
  • rs/moq-auth/rate.json
  • rs/moq-auth/src/claims.rs
  • rs/moq-auth/src/grant.rs
  • rs/moq-auth/src/key.rs
  • rs/moq-auth/src/lib.rs
  • rs/moq-auth/src/rate.rs
  • rs/moq-auth/src/serve.rs
  • rs/moq-auth/src/set.rs
  • rs/moq-auth/src/wire.rs
  • rs/moq-auth/tests/released_token.rs
  • rs/moq-net/CHANGELOG.md
  • rs/moq-net/src/transport.rs
  • rs/moq-relay/CHANGELOG.md
  • rs/moq-relay/src/auth.rs
  • rs/moq-relay/src/auth/caps.rs
  • rs/moq-relay/src/auth/refresh.rs
  • rs/moq-relay/src/cluster.rs
  • rs/moq-relay/src/connection.rs
  • rs/moq-relay/src/incoming.rs
  • rs/moq-relay/src/relay.rs
  • rs/moq-relay/src/uring.rs
  • rs/moq-relay/tests/integration/bitrate_caps.rs
  • rs/moq-relay/tests/integration/main.rs
  • rs/moq-relay/tests/integration/peer_limits.rs
  • rs/moq-tokio/CHANGELOG.md
  • rs/moq-tokio/src/quic/backend.rs
  • rs/moq-tokio/src/server.rs
Files with no reviewable changes: 3
  • quest/m2/README.md
  • quest/m2/quic-sim.md
  • quest/m2/rate-quic.md

Included review availability: This review used your included allowance. Your plan provides up to 4 included reviews per hour; 2 remain after this review.

Walkthrough

The change adds publish and subscribe bitrate caps to Rust and JavaScript authentication claims and grants. Relay authorization combines caps across grants and applies them through pacers on supported sessions; transports without enforcement refuse capped grants. QUIC servers use setup-sized receive credit before admission, then apply session limits. New tests cover serialization, cap updates, transfer pacing, and receive-credit changes.

Estimated code review effort

Merge Risk: Minimal · up to b9fb2

No merge-blocking issue is established; the change is ready for normal checks.

Pre-merge checks: Passed 4 · Failed 1

Failed Failed checks: 1 warning

Check name Status Explanation Resolution
Docstring Coverage Warning Docstring coverage is 59.02% which is insufficient. The required threshold is 80.00%. Docstring coverage is scoped to functions touched by this diff with cyclomatic complexity of at least 2. Documenta… Write docstrings for the functions missing them to satisfy the coverage threshold.
Passed Passed checks: 4 passed
Check name Status Explanation
Title check Passed The title clearly and concisely identifies the main change: enforcing bitrate caps on QUIC relay sessions.
Description check Passed The description directly explains the auth, relay enforcement, transport behavior, pre-auth window, tests, and related implementation changes.
Linked Issues check Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check Passed Check skipped because no linked issues were found for this pull request.
Full details: Docstring Coverage

Explanation

Docstring coverage is 59.02% which is insufficient. The required threshold is 80.00%. Docstring coverage is scoped to functions touched by this diff with cyclomatic complexity of at least 2. Documentation-only edits and deletions within surviving functions are included; unchanged diff context and fully deleted functions are excluded. Analyzed 61 functions across 32 files. (14 skipped: 14 unsupported.)

Resolution

Write docstrings for the functions missing them to satisfy the coverage threshold.

Finishing Touches
Simplify code
  • Commit to this branch
  • Create a new PR
  • Autofix · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

Warning

Some tools did not complete. Review the errors below.

Biome (2.5.15)
js/auth/src/claims.test.ts

Biome could not lint this file: configuration resulted in errors. Check the repository's Biome configuration and plugins.

js/auth/src/claims.ts

Biome could not lint this file: configuration resulted in errors. Check the repository's Biome configuration and plugins.

js/auth/src/contract.test.ts

Biome could not lint this file: configuration resulted in errors. Check the repository's Biome configuration and plugins.

  • 7 others
Clippy (1.99.0)

Clippy execution timed out

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

Share

@kixelated kixelated left a comment

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Automated review by review (OpenAI)

Reviewed commit: 07316c1cdc363906a5fa3bd8749639ccea84a0bd.

No actionable bugs found in the changed auth encoding, admission, credential-union/revalidation, and Tokio/io_uring pacing paths. No earlier published review findings to carry forward.

Direction is sound: reuse the existing QUIC limiter, apply caps before opening receive credit, and refuse capped grants where enforcement is unavailable. Keeping auth policy separate from transport pacing is simpler than introducing another payload-rate wrapper; the worker handoff is justified by io_uring's thread affinity. Strict object-role encoding preserves caps instead of silently downgrading them for legacy verifiers.

Verification: GitHub source and test inspection, including an independent Rust/JS auth-wire pass. I did not run builds, tests, interop, or throughput measurements; the PR's reported results were not independently reproduced.

(Written by OpenAI)

@pullfrog pullfrog Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Important

The upgrade and migration guides still document the previous behavior: that object roles and bitrate caps are refused. This PR re-enables them, so those entries now contradict the code.

Reviewed changes

  • moq-auth caps. New Rate / InvalidRate, additive publish_rate / subscribe_rate on Claims and Grant, and the {"path": [...], "rate": bps} wire role in wire::Role, which released verifiers refuse rather than drop the cap. Key scopes stay path-only. serve copies a token's caps into its grant.
  • js/auth mirror. WireRoleSchema, decodeRoles / nestRates / encodeRoles, encodeGrant, and caps on Key.sign; GrantSchema becomes a pipe (its .shape is gone).
  • Relay enforcement (auth/caps.rs). Caps + Pacer hold a session's QUIC connection to the union of its live credentials' caps; Auth::admit refuses a capped grant without a pacer, admit_paced carries one, and revalidation/in-band credentials pace the live connection.
  • Pre-auth window. moq_net::transport::SETUP_RECEIVE_WINDOW and moq-tokio's Request::with_quic_limits; every server QUIC connection holds the SETUP window until ok opens [quic] receive_window.
  • Tests. bitrate_caps.rs on the tokio and io_uring listeners, auth::caps / auth::refresh / auth, shared rate.json vectors on both sides, and the updated peer_limits.

⚠️ Upgrade and migration guides still say caps are refused

This PR reverses the Unreleased "Auth roles are patterns" change: the object role {"path": [...], "rate": bps} is accepted again and Rate / InvalidRate are back. Four hand-written guides still state the opposite, and the moq-tokio guide is missing a note for the new pre-auth window. A deployer following the upgrade page would believe caps are unsupported.

Technical details
# Stale upgrade/migration prose after re-enabling caps

## Affected sites
- `doc/setup/upgrade.md:60-62` — "Auth roles are pattern arrays ... An object role, including one with a `rate` cap, is refused; use pattern arrays for `publish` and `subscribe`."
- `rs/moq-auth/MIGRATE.md:56` — "`Access`, `Rate`, and `InvalidRate` are removed; bitrate restrictions are unsupported and the whole object role is refused, including `{"path":["alice/**"],"rate":8000000}`."
- `rs/moq-relay/MIGRATE.md:125` — "Token and auth-server grant parsers refuse object roles, including bitrate caps, before relay admission; no transport currently enforces a cap."
- `js/auth/MIGRATE.md:21` — "Bitrate restrictions are unsupported and the whole object role is refused, including `{"path":["alice/**"],"rate":8000000}`."
- `rs/moq-tokio/MIGRATE.md` (`## Unreleased`) — no entry for `server::Request::with_quic_limits` or the window held until `ok`.

## Required outcome
- Each "Auth roles are patterns" entry describes the net behavior on `main`: caps are supported as
  `publish_rate` / `subscribe_rate` beside the patterns, the capped object role is accepted, and the
  `Access` wrapper is still gone.
- `doc/setup/upgrade.md` carries the rollout order the new wire implies: a capped role is refused by
  older relays/auth servers rather than silently dropped, so upgrade auth servers and relays before
  minting capped tokens.
- `rs/moq-tokio/MIGRATE.md` notes the behavior change already in its CHANGELOG: a server QUIC
  connection holds `SETUP_RECEIVE_WINDOW` until `Request::ok`, so a caller that set limits with
  `Request::quic().set_limits(..)` before `ok` should use `Request::with_quic_limits` instead.

## Suggested approach (optional)
- Fold the changes into the existing `## Unreleased` sections rather than adding new ones, matching the
  CHANGELOG entries this PR already added under `### Added` / `### Changed`.

## Open questions for the human (optional)
- If these guides are intentionally written at release cut (per `doc/setup/upgrade.md`), say so and
  close this; otherwise they contradict the CHANGELOGs in this same PR.

Pullfrog  | Fix it ➔ | View workflow run | Using deepseek-v4.1-flash (free via Pullfrog for OSS) | 𝕏

@kixelated
kixelated enabled auto-merge (squash) November 1, 2026 22:51
@kixelated
kixelated added this pull request to the merge queue Nov 2, 2026
@github-merge-queue
github-merge-queue Bot removed this pull request from the merge queue due to a conflict with the base branch Nov 2, 2026
@moq-bot

moq-bot Bot commented Nov 2, 2026

Copy link
Copy Markdown
Contributor

quest land watch: the merge queue removed this pull request at 07316c1 because it conflicts with main (merge_conflict).

The land fixer works on it in the background, and the watcher re-enqueues it at the back of the queue once it is fixed or judged flaky, or hands it back here with what it needs.

(written by quest land watch)

@moq-bot
moq-bot Bot enabled auto-merge November 2, 2026 08:58
@moq-bot

moq-bot Bot commented Nov 2, 2026

Copy link
Copy Markdown
Contributor

codex / Codex land fixer note:

Merged base into the pinned PR head and resolved the initializer conflict by retaining the PR receive-window field and the base peer-watcher field. No additional behavior changes. Rust Nextest validation passed 4,535 tests with 37 skipped, including relay io_uring coverage. JavaScript auth type checks, all 128 tests, and the package build passed. The initial cargo test --locked -p path+file:///work/rs/moq-auth -p path+file:///work/rs/moq-net -p path+file:///work/rs/moq-tokio -p path+file:///work/rs/moq-relay --features moq-relay/io-uring hit a shared-process tracing subscriber collision in setup::tests::credentials_stay_out_of_traces: a global default trace dispatcher has already been set. That test passed alone with cargo test --locked -p path+file:///work/rs/moq-net --lib setup::tests::credentials_stay_out_of_traces and in the full isolated Nextest run.

(written by codex / Codex)

@moq-bot

moq-bot Bot commented Nov 2, 2026

Copy link
Copy Markdown
Contributor

quest land watch: armed auto-merge on this pull request at b9fb280, so it joins the back of the merge queue once its required checks pass, with the land fixer's fix (codex / Codex): Merged base and retained both QUIC Server initializer fields; passed cargo nextest run --locked -p path+file:///work/rs/moq-auth -p path+file:///work/rs/moq-net -p path+file:///work/rs/moq-tokio -p path+file:///work/rs/moq-relay --features moq-relay/io-uring; bun run --cwd js/auth check; bun run --cwd js/auth test; bun run --cwd js/auth build; rustfmt --check --edition 2024 rs/moq-tokio/src/quic/backend.rs; git diff --cached --check..

(written by quest land watch)

@moq-bot
moq-bot Bot added this pull request to the merge queue Nov 2, 2026
Merged via the queue into main with commit b51eb8c Nov 2, 2026
10 checks passed
@moq-bot
moq-bot Bot deleted the quest/m2/rate-quic branch November 2, 2026 10:45
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant