Skip to content

feat(kio)!: generate async wrappers behind an async feature - #7506

Merged
moq-bot[bot] merged 17 commits into
mainfrom
quest/m1/poll-first/async-feature
Nov 1, 2026
Merged

moq-bot[bot] merged 17 commits into
mainfrom
quest/m1/poll-first/async-feature

Conversation

@kixelated

@kixelated kixelated commented Nov 1, 2026 •

Copy link
Copy Markdown
Collaborator

Problem

The async-feature quest needed a generator before rollout. This PR first held the spike that compared three generators (results). The maintainer has now chosen one, so the PR builds it and makes the first conversions.

Decisions

The maintainer chose this on 2026-11-01:

  1. ✅ Method attribute (recommended): reuse kio-macros, keep the poll source as ordinary Rust, derive the wrapper's name, signature, and docs, and accept the measured cost of a standalone kio build (about 3.5 s clean).
  2. Declarative macro: the lowest standalone kio build cost, but it needs its own declaration syntax and both method names spelled out.
  3. rust2-driver: a larger nightly compiler spike before choosing.

Choices I made while implementing it, listed here for review:

  • Callers gate the attribute with #[cfg_attr(feature = "async", kio::wrapper)], so the macro emits no cfg itself. A poll-only build then never runs the proc macro, the macro doesn't depend on what a caller names its feature, and the attribute is testable inside kio-macros. The spike had the macro also emit #[cfg(feature = "async")], which would have checked kio-macros' own features in its tests. The CI scan below catches any wrapper left ungated.
  • Docs are copied from the poll, then a line Awaits [poll_x](Self::poll_x). is added. The poll keeps its own docs, so they now say what the call resolves to rather than how it registers a waiter. I rewrote kio's and broadcast's poll docs to read that way. The link only goes from the wrapper to the poll, because a link back would break when async is off.
  • Contract: the receiver is &self or &mut self. There is exactly one &Waiter argument, in any position. Every other argument has a plain name and is passed to every poll unchanged, so it must be Copy or a reference; rustc points at the argument otherwise. The return type is Poll<T>. The wrapper copies the poll's visibility, generics (named in a turbofish), doc, cfg, and deprecated. A poll named after a keyword (poll_match) gets a raw identifier. I checked this against every direct wrapper in kio and moq-net: they include waiter-last polls, &mut arguments, polls generic only in the return type, and method where clauses.
  • Composes with #[kio::machine], which a test covers.
  • kio's own async helpers that do more than await one poll stay hand-written and gated: Queue::push, the wait predicates, clock::drive, and Interval::tick. sim enables async, because the simulator only runs async tests.
  • The spike fixtures are removed. Its runner appended its own wrapper to kio-macros, which now conflicts, and the results stay linked by commit.

Approach

  • #[kio::wrapper] lives in kio-macros (rs/kio/macros/src/wrapper.rs), and kio re-exports it under macros.
  • kio has a new default-off async feature (implies macros). Without it, kio compiles no coroutines: Consumer/Producer/ConsumerWeak/ProducerWeak::closed, Producer::wanted_changed, and Queue::pop are generated, and the rest are gated.
  • moq-net's async now enables kio/async. moq-net's broadcast wrappers are generated: Requests::{requested_track, closed}, Consumer::closed, and Demand::wanted_changed.
  • sh/rs/coroutines.sh emits a library's MIR and fails on any async fn body, block, or closure not on its allowlist. It also fails when an allowed name disappears, so the list can only shrink. just rs coroutines runs it on kio's and moq-net's poll-only builds, and just rs features (nightly and release PRs) runs that, following the rule that isolated feature builds stay out of the gate. moq-net's poll-only build still compiles three: auth::until, and origin's run_front and serve_front.
  • Consumer wiring: moq-shaper's binary and moq-auth's library are the only crates that await kio directly without reaching kio/async through moq-net, so each enables it. I checked every direct kio dependent's normal-dependency graph with cargo tree -e features,normal, then built each crate that lacked it as a lone library. just rs features now checks moq-auth's lone library build, because the workspace's kio/sim dev-dependencies unify kio/async back in and hid the break (Pullfrog's finding).
  • A poll whose wrapper name cannot be an identifier, even raw (poll_self, poll__, poll_1), gets a spanned error instead of a proc-macro panic (Pullfrog's finding), with a UI case.
  • rs/AGENTS.md's rule now says the async helper is generated and gated.
  • The quest is rewritten to list what's left: moq-net's ~47 other wrappers, the three poll-only coroutines, the async logic bodies, the sibling libraries, and turning moq-net's async default off.

Impact

  • Breaking for kio's public API. Its async methods need features = ["async"]. rs/kio/MIGRATE.md has the migration note. Version bumps stay manual.
  • kio-macros adds the wrapper attribute (additive).
  • moq-net's API, default features, wire format, bindings, and JS are unchanged.

Checks

  • cargo clippy -D warnings on kio, for each of runtime, macros, async, runtime,manual,serde, and async,runtime.
  • cargo doc -D warnings on kio with no features, with poll-only features, and with all features; on moq-net with default features and with none.
  • cargo test -p kio -p kio-macros, doctests included.
  • just rs coroutines scans (kio: none; moq-net: the three allowed), plus a negative run that confirms the scan catches kio's coroutines with async on.
  • sh/rs/net-features.sh: 1279 tests passed. sh/rs/tokio-features.sh passed.
  • The standalone --no-default-features checks from just rs features: moq-relay, moq-cli, moq-bench, moq-boy, and moq-hls/rtmp/rtc with all targets.
  • Standalone --all-targets checks of moq-auth, moq-congestion, moq-ice, moq-quic, moq-shaper, and moq-net-tests.
  • just ci origin/main (with a scratch CARGO_TARGET_DIR and BASH_ENV=/dev/null) passed every module except check flake. That one fails while building the github-runner VM derivation: unshare: write failed /proc/self/uid_map: Operation not permitted. The cause is this host's sandbox; the PR changes no Nix files. Rust: 10934 tests passed. The first run caught a shellcheck SC1087 in sh/rs/coroutines.sh, which is now fixed.
  • quest check --base origin/main passed.
  • After the Pullfrog fixes: cargo test on kio-macros (UI cases included) and kio with all features, cargo clippy -p kio -p kio-macros --all-targets --all-features -D warnings, just rs coroutines, quest check, and lone --lib checks of moq-auth (all features), moq-ice, moq-congestion, and moq-quic. moq-auth's failed before the fix with no method named wait found for struct Shared<T>.
  • After merging origin/main 3ae6789 (head 1702e940): just rs coroutines (kio: none; moq-net: the three allowed), cargo test -p kio -p kio-macros --all-features (UI cases and doctests included), cargo clippy -p kio -p kio-macros --all-targets --all-features -D warnings, cargo clippy -p kio -D warnings with no features and with each of macros, async, runtime,manual,serde, and async,runtime, and quest check --base origin/main. just rs features passed every step but its last, the workspace nextest. There, 11070 of 11071 tests passed (rerun with --no-fail-fast). The one failure, moq-relay::integration stack::debug_relay_load_and_run_fit_windows_stack, overflows its 1 MiB stack under --all-features. It fails the same way on main at 3ae6789 without this PR, and moq-dev/moq#7576 already plans it as quest/m0/test-flakes-2/relay-stack-all-features.md.

Review

Codex harness, gpt-6-astra, medium effort (--deep, because kio's public API changes), against origin/main 75c48bb:

  • Round 1, head ebd7b35276d8d5ae948b70eb7501815fb2902b69: no findings.
  • Round 2, head 398809e3602bc1658ef6ab307e2c16a9b0a66eac (the shellcheck fix and a conflict-free merge of origin/main): no findings.

Head 1f79714e4f42ff065929bf5488471527a902d23c (a conflict-free merge of origin/main 2ebcea5 and the two Pullfrog fixes): local review skipped under the simple-review rule (moq-dev/quest#364). Both fixes are a few lines and touch no public API, wire format, security code, or unsafe: a diagnostic in place of a proc-macro panic, and a cargo feature plus one just rs features check. The online reviewers cover this head.

Head 1702e940966f5929b7f0b695a1c500ffa9240320 (a merge of origin/main 3ae6789): local review skipped under the simple-review rule, since the merge is mechanical. Its one conflict was in rs/kio/src/clock/mod.rs's re-exports. The resolution keeps both sides unchanged: main's new pub use random::{Rng, rng, seed}; and this PR's #[cfg(feature = "async")] pub use timer::drive;. main's new kio code (the seeded clock::rng) and its callers in moq-net, moq-relay, moq-hls, moq-rtmp, moq-tokio, moq-audio, and moq-video add no async kio API and await nothing newly gated. Its only new kio async code is a test helper in clock::sim's tests, which builds with sim, and sim already enables async. No online-reviewer thread is open.

Pullfrog on 398809e:

  • moq-auth's library build breaks without kio/async: fixed.
  • Ident::new_raw panics for poll_self and poll__: fixed, and for poll_1 too.

Follow-ups

The rewritten quest covers the remaining rollout. Suggested for /quest-plan: split the quest into children, one each for moq-net's remaining wrappers, the origin front driver poll machine, and each sibling library's async feature, since each can land on its own.

(Written by Claude Opus 5.5)

kixelated and others added 8 commits October 31, 2026 23:42
Co-Authored-By: GPT-6.1 Sol <noreply@openai.com>
Co-Authored-By: GPT-6.1 Sol <noreply@openai.com>
Co-Authored-By: GPT-6.1 Sol <noreply@openai.com>
Co-Authored-By: GPT-6.1 Sol <noreply@openai.com>
Co-Authored-By: GPT-6.1 Sol <noreply@openai.com>
Co-Authored-By: GPT-6.1 Sol <noreply@openai.com>
…oice

Co-Authored-By: GPT-6.1 Sol <noreply@openai.com>
@kixelated kixelated changed the title Generated async wrappers behind a feature chore(kio): checkpoint the async wrapper generator spike Nov 1, 2026
kixelated and others added 4 commits November 1, 2026 10:24
The maintainer picked the method attribute from the generator spike.
`#[kio::wrapper]` on a `poll_*` method adds the `async fn` that awaits it,
deriving its name, signature, generics, and docs. Its contract is covered by
kio-macros' behavior and UI tests.

kio's async methods move behind a new default-off `async` feature, so a
poll-only kio compiles no coroutines. moq-net's `async` enables it, and
moq-net's broadcast wrappers become generated. `just rs coroutines` scans the
MIR of kio's and moq-net's poll-only builds in the nightly feature matrix.

The spike fixtures are removed: the runner appended its own `wrapper` to
kio-macros, and its results stay linked from the quest by commit.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@kixelated kixelated changed the title chore(kio): checkpoint the async wrapper generator spike feat(kio)!: generate async wrappers behind an async feature Nov 1, 2026
@kixelated
kixelated marked this pull request as ready for review November 1, 2026 18:54
@coderabbitai

coderabbitai Bot commented Nov 1, 2026 •

Copy link
Copy Markdown
Contributor

Warning

Review limit reached

You've used all free OSS reviews for now. Wait for the free limit to reset to keep reviewing this public repository.

Next included review available in 10 minutes.

Check out review usage here.

View limit details

Limit details: You’ve used all 4 included reviews currently available.

Learn how review limits work.

Review configuration:

Run configuration
  • Source mode: Auto
  • Repository: Unavailable
  • Branch: Unavailable
  • File: No configuration file from this repository applied
  • Commit: Unavailable
  • Layers: Organization UI
  • Review profile: CHILL
  • Plan: Advanced
  • Run ID: 37610ed2-8191-4bee-9a63-feff28cf3da7
Commits

Reviewing files that changed from the base of the PR and between 3ae6789 and 1702e94.

Files selected for processing: 39
  • quest/m1/poll-first/async-feature.md
  • rs/AGENTS.md
  • rs/justfile
  • rs/kio/Cargo.toml
  • rs/kio/MIGRATE.md
  • rs/kio/macros/src/lib.rs
  • rs/kio/macros/src/wrapper.rs
  • rs/kio/macros/tests/ui/wrapper_moved_argument.rs
  • rs/kio/macros/tests/ui/wrapper_moved_argument.stderr
  • rs/kio/macros/tests/ui/wrapper_name.rs
  • rs/kio/macros/tests/ui/wrapper_name.stderr
  • rs/kio/macros/tests/ui/wrapper_no_receiver.rs
  • rs/kio/macros/tests/ui/wrapper_no_receiver.stderr
  • rs/kio/macros/tests/ui/wrapper_output.rs
  • rs/kio/macros/tests/ui/wrapper_output.stderr
  • rs/kio/macros/tests/ui/wrapper_owned_receiver.rs
  • rs/kio/macros/tests/ui/wrapper_owned_receiver.stderr
  • rs/kio/macros/tests/ui/wrapper_pattern.rs
  • rs/kio/macros/tests/ui/wrapper_pattern.stderr
  • rs/kio/macros/tests/ui/wrapper_qualifiers.rs
  • rs/kio/macros/tests/ui/wrapper_qualifiers.stderr
  • rs/kio/macros/tests/ui/wrapper_unnameable.rs
  • rs/kio/macros/tests/ui/wrapper_unnameable.stderr
  • rs/kio/macros/tests/ui/wrapper_waiter.rs
  • rs/kio/macros/tests/ui/wrapper_waiter.stderr
  • rs/kio/macros/tests/wrapper.rs
  • rs/kio/src/clock/mod.rs
  • rs/kio/src/clock/timer.rs
  • rs/kio/src/consumer.rs
  • rs/kio/src/lib.rs
  • rs/kio/src/producer.rs
  • rs/kio/src/queue.rs
  • rs/kio/src/shared.rs
  • rs/kio/src/weak.rs
  • rs/moq-auth/Cargo.toml
  • rs/moq-net/Cargo.toml
  • rs/moq-net/src/model/broadcast.rs
  • rs/moq-shaper/Cargo.toml
  • sh/rs/coroutines.sh
  • Autofix · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

Share

@kixelated kixelated left a comment

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Automated review by review (OpenAI)

Reviewed commit: 398809e

No actionable bugs found. Reusing kio-macros keeps ordinary poll methods canonical without introducing a second declaration language. The generated wrappers preserve the converted methods' polling behavior; feature forwarding and the explicit migration note match the default-off kio API change. Keeping the nontrivial helpers handwritten is appropriate, and the isolated coroutine scan complements the wrapper tests.

Verification: static inspection of the generator, conversions, feature wiring, tests and scan script only. I did not run builds/tests, reproduce the compile-time measurements, or independently verify the reported feature matrix.

(Written by OpenAI)

@pullfrog pullfrog Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Important

moq-auth's library build breaks with this change: it calls kio::Shared::wait, now gated behind kio/async, but never enables that feature. The workspace build hides this through feature unification, so it slips past just ci and just rs features.

Reviewed changes

  • #[kio::wrapper] macro — kio-macros derives an async fn beside a poll_* method, copying the poll's visibility, generics (turbofish'd), docs, cfg, and deprecation, plus a back-link, and errors with a compile error beside the original method when the shape is rejected. Covered by behavior tests and trybuild UI diagnostics.
  • kio async feature — a new default-off feature (implies macros) gates every kio async method; sim now enables async; extern crate self as kio widens to any(test, feature = "async"); MIGRATE.md, Cargo.toml, doctests, and module docs updated.
  • moq-net wrappers — Requests::{requested_track, closed}, Consumer::closed, and Demand::wanted_changed are now generated from their polls, and moq-net's async enables kio/async.
  • just rs coroutines — sh/rs/coroutines.sh scans a crate's MIR and fails on any coroutine not on its shrinking allowlist; kio allows none, moq-net allows until/run_front/serve_front. Wired into just rs features.
  • Feature wiring and docs — moq-shaper gains kio/async; rs/AGENTS.md, the async-feature quest, and rs/kio/MIGRATE.md describe the generated-and-gated rule.

⚠️ moq-auth's library build no longer compiles without kio/async

Gating Shared::wait behind kio/async leaves moq-auth without the feature it needs, and moq-auth depends on kio directly rather than through moq-net, so nothing pulls the feature in. With resolver 2, a non-test build resolves kio from normal dependencies only (cargo tree -e features,no-dev -p moq-auth shows default + serde, no async), so a plain cargo check -p moq-auth and any downstream consumer of the published crate fail. The workspace gate and the PR's --all-targets check both unify kio/async in (the sim dev-dependency enables it), which is why CI stays green.

Technical details
# moq-auth needs the kio `async` feature

## Affected sites
- `rs/moq-auth/Cargo.toml:25` — `kio = { workspace = true, features = ["serde"] }` enables no `async`.
- `rs/moq-auth/src/lease.rs:416` — `Shared::wait` in `Lease::reported`.
- `rs/moq-auth/src/client.rs:215` — `Shared::wait` on `connection.lifecycle` (`kio::Shared<Lifecycle>`).
- `rs/moq-auth/src/client.rs:528` — `Shared::wait` on the `Log`'s `kio::Shared<Vec<Request>>`.

## Required outcome
- `cargo check -p moq-auth` (library, default features and `--no-default-features`) compiles.

## Suggested approach
- Add `"async"` to `moq-auth`'s `kio` features, as `moq-shaper` did, until it grows its own `async` feature per the quest.
- Add an isolated lib-only check for the kio-direct crates, since both the workspace gate and `--all-targets` unify `kio/async` through the `sim` dev-dependency and mask the gap.

## Open questions for the human
- Should `just rs coroutines` list `moq-auth` with `--allow` now, or wait for its `async` feature?

Pullfrog  | Fix all ➔ | Fix 👍s ➔ | View workflow run | Using deepseek-v4.1-flash (free via Pullfrog for OSS) | 𝕏

Comment thread rs/kio/macros/src/wrapper.rs Resolved
kixelated and others added 3 commits November 1, 2026 11:42
`poll_self`, `poll__`, and `poll_1` reached `Ident::new_raw`, which panics for
the keywords that cannot be raw and for a digit-led name. Report a spanned
error on the poll instead.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
moq-auth depends on kio directly and awaits `Shared::wait`, now behind
`kio/async`. The workspace build unified the feature in through the `kio/sim`
dev-dependencies, so only a lone library build broke. `just rs features` now
checks that build.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

@kixelated kixelated left a comment

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Automated review by review (OpenAI)

Reviewed commit: 1f79714

No new actionable bugs found since my previous review. The explicit moq-auth kio/async dependency fixes the isolated-library build issue identified in Pullfrog's review, and the lib-only feature check targets the feature-unification blind spot. The wrapper now reports a spanned error for unnameable identifiers instead of panicking, addressing the existing finding.

These are focused fixes that preserve the generator's direction without extra abstraction. Base-branch changes were separated from this update. Verification was static inspection only; I did not run the build matrix or UI tests.

(Written by OpenAI)

@pullfrog pullfrog Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

✅ No new issues found.

Reviewed changes

Re-reviewed the commits since the prior Pullfrog review on 398809e, which had raised two issues. Both are fixed correctly, and the new code introduces nothing else.

  • moq-auth library build — rs/moq-auth/Cargo.toml enables kio/async on its kio dependency, so Shared::wait resolves in an isolated library build, and just rs features adds a lone --lib --all-features moq-auth check to guard the feature-unification blind spot.
  • Unnameable wrapper names — #[kio::wrapper] emits a spanned error for a name that is no identifier even raw (poll_self, poll__, poll_1) instead of panicking in Ident::new_raw; the guard set (_, self, Self, super, crate) matches proc-macro2's raw-identifier restrictions exactly, and tests/ui/wrapper_unnameable covers all three cases.

Pullfrog  | View workflow run | Using deepseek-v4.1-flash (free via Pullfrog for OSS) | 𝕏

Resolve the clock re-export conflict: keep main's random exports and the
async-gated drive.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@kixelated
kixelated enabled auto-merge (squash) November 1, 2026 21:12
@moq-bot
moq-bot Bot merged commit 7a48eb3 into main Nov 1, 2026
1 check passed
@moq-bot
moq-bot Bot deleted the quest/m1/poll-first/async-feature branch November 1, 2026 21:58
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant