diff --git a/AGENTS.md b/AGENTS.md index 6eb735453..8797700d1 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -1,14 +1,16 @@ -rust2 holds Rust tools that take Rust somewhere else: to TypeScript through an own rustc driver, to any language through a fork of UniFFI, and to faster builds for a pure-Rust subset. +rust2 holds Rust tools that take Rust somewhere else: to any language through an exporter of a crate's `pub` API, which replaces a fork of UniFFI, to TypeScript through an own rustc driver, and to faster builds by measuring upstream's levers. Each project lives in its own top-level directory; the [quest tree](quest/README.md) says what each is doing now. # Required - An unsupported construct or malformed input is a compile error with a source span, never a warning, a silent fallback, or a runtime failure. Generated code panics only where the Rust source does. - The one exception is an `unsafe` pointer from elsewhere whose origin translation can't see, such as a parameter's: a loud run-time check may stand in for the refusal, under m3's [unsafe policy](quest/m3/README.md), since refusing every such pointer would refuse pointer arithmetic on any parameter. + One exception is an `unsafe` pointer from elsewhere whose origin translation can't see, such as a parameter's: a loud run-time check may stand in for the refusal, under m3's [unsafe policy](quest/m3/README.md), since refusing every such pointer would refuse pointer arithmetic on any parameter. + The other is a binding, which checks at run time only the Rust rules its language cannot express: borrow exclusivity, use after a move or close, and the thread of a value that is not `Send`. A failed check raises the language's own error, or aborts where it has none; a Rust panic aborts. - Generated output must match the source's semantics exactly: integer widths and overflow, drop order, laziness, and error paths. Prove it with differential tests against native Rust, not by inspection. +- A binding's types are lossless: every value of the Rust type crosses and returns exactly. A native type stands in only where it is exact; otherwise the runtime's own type crosses, with checked conversions to and from the idiomatic one. Speed ranks above ergonomics. - Everything is configless, with opinionated names and defaults. Translators take stock crates as they are: no annotations, and no config beyond true platform boundaries. Each config key or source change a consumer needs is a gap to close. - A vanity rename, if ever supported, is opt-in and lives inline in source next to the item, never in a config file. -- moq is the first consumer of every project here. +- moq is the first consumer of every project here; a binding target may be planned before any consumer asks for it. - Prefer upstream and maintained crates over hand-rolling. Measure before claiming an improvement over them. # Development diff --git a/README.md b/README.md index 80b73e569..222a36ea4 100644 --- a/README.md +++ b/README.md @@ -2,10 +2,10 @@ Rust to other places. -- **TypeScript**: an own rustc driver that translates a [documented Rust subset](ts/README.md) to readable TypeScript, without WASM. +- **TypeScript**: an own rustc driver that translates a [documented Rust subset](ts/README.md) to TypeScript, without WASM. - **C, then any language**: an [exporter](ffi/README.md) that turns a crate's `pub` API into a C ABI and header with no attributes, sharing the TypeScript driver's front end. - **Any language**: [a hard fork](uniffi/README.md) of [UniFFI](https://github.com/mozilla/uniffi-rs) with every generator [moq](https://github.com/moq-dev/moq) uses. -- **Faster builds**: a reproducible [build-speed benchmark](bench/build/README.md), then a pure-Rust subset (no build scripts, no C linking) that builds faster, and [rust2-lint](driver/src/bin/rust2-lint/main.rs), which fails a build whose generics' copies cost its codegen more than a budget (`nix run github:moq-dev/rust2#lint -- --help`). +- **Faster builds**: a reproducible [build-speed benchmark](bench/build/README.md) of upstream's levers on moq, and [rust2-lint](driver/src/bin/rust2-lint/main.rs), which fails a build whose generics' copies cost its codegen more than a budget (`nix run github:moq-dev/rust2#lint -- --help`). Everything here is early; the [quest tree](quest/README.md) holds the plan. diff --git a/quest/README.md b/quest/README.md index 0ffb28aa1..8e72f65f5 100644 --- a/quest/README.md +++ b/quest/README.md @@ -7,14 +7,21 @@ grouped into milestones ordered by priority. ## Plan -m0 holds urgent fixes. m1 is the work with a consumer waiting on it now: -the exporter that replaces the UniFFI fork (moq skips switching to the fork). m2 is -research: the TypeScript translator, which moq's browser package adopts once it matches -the handwritten one, and the fork's gated or speculative work. m3 holds work that waits -on evidence or has no consumer yet: the build-speed tracks, which the -[build-speed benchmark](/bench/build/README.md) rescopes before they start, and the -parts of std and the translators' features no consumer calls yet. m4 holds long-term -research. +m0 holds urgent fixes and what every run pays for. m1 is the work with a consumer +waiting on it now: the exporter moq switches its bindings to. m2 is research with a named +consumer: rust2-wasm, moq's browser route, and the TypeScript translator, an experiment +that has to beat it. m3 holds work that waits on evidence or has no consumer yet: +upstream's build-speed levers, which the [build-speed benchmark](/bench/build/README.md) +measures, the translator's features that wait on that gate, and the binding targets +planned ahead of a consumer. m4 holds long-term research. + +Decided with the maintainer on 2026-11-09, in the quest audit, the goal: one Rust code +base callable from any language or framework, with no manual wrappers or annotations. +Speed ranks above API ergonomics, Rust's correctness is kept wherever the target can hold +it, and what is unsupported fails the export, not the run. The +[exporter](/quest/m1/ffi-export/README.md) is the product, WebAssembly its browser +target, the translator an experiment against that, and build speed a side track that +keeps only what costs nothing. Decided with the maintainer on 2026-10-23, when rust2 was planned from moq ([moq-dev/moq#6578](https://github.com/moq-dev/moq/pull/6578)): @@ -29,8 +36,8 @@ Decided with the maintainer on 2026-10-23, when rust2 was planned from moq ## Required -- [m0: immediate priorities](/quest/m0/README.md) - urgent fixes -- [m1: consumers waiting](/quest/m1/README.md) - the exporter that replaces the UniFFI fork for moq -- [m2: research](/quest/m2/README.md) - the TypeScript translator's spikes and features toward moq-net's model, and the fork's gated work -- [m3: deferred](/quest/m3/README.md) - upstream's build-speed levers, measured by the benchmark, and the std gaps and translator features no consumer calls yet +- [m0: immediate priorities](/quest/m0/README.md) - urgent fixes, a green nightly, and the fork's deletion +- [m1: consumers waiting](/quest/m1/README.md) - the exporter moq can pin and switch its bindings to +- [m2: research](/quest/m2/README.md) - rust2-wasm as the browser route, and the TypeScript translator that has to beat it +- [m3: deferred](/quest/m3/README.md) - upstream's build-speed levers, the translator features that wait on the gate, and binding targets planned ahead of a consumer - [m4: long-term research](/quest/m4/README.md) - ideas no consumer needs yet, such as a rustc and cargo fork diff --git a/quest/m0/README.md b/quest/m0/README.md index fa2096325..ab5c5f471 100644 --- a/quest/m0/README.md +++ b/quest/m0/README.md @@ -2,8 +2,12 @@ ## Goal -Urgent fixes to anything a consumer already depends on. +Urgent fixes to anything a consumer already depends on, and what every agent pays for on +each run: the checks, a red nightly, and code no consumer uses. ## Required - [Checks at moq's cost](/quest/m0/checks/README.md) - `just ci` selects by the diff, builds warm, and runs the differential optimized, with a nightly full `just gate` +- [Delete the UniFFI fork](/quest/m0/delete-uniffi.md) - `uniffi/`, its workspace, flake packages, nightly jobs, and the benchmark harness built on it are gone +- [The nightly passes](/quest/m0/nightly-green.md) - red since 2026-10-29: the wasm-bindgen fixture's NaN, Kotlin on Android API 35, and `just ffi moq` +- [Trim the build-speed benchmark](/quest/m0/bench-trim.md) - it runs on a pin change, not nightly, without the nix job and the `pure` corpora diff --git a/quest/m0/bench-trim.md b/quest/m0/bench-trim.md new file mode 100644 index 000000000..0ec31739c --- /dev/null +++ b/quest/m0/bench-trim.md @@ -0,0 +1,29 @@ +# [S] Trim the build-speed benchmark + +## Goal + +The [build-speed benchmark](/bench/build/README.md) costs CI only when its result can +change: `bench.yml` runs when a pin or the benchmark itself changes and on manual +dispatch, not nightly; the `Bench (moq nix)` job and `bench/build/nix.*` are deleted, and +so are the `pure` corpora (regex, wasmparser) and their job, with every table they +produced kept in the README. + +## Plan + +Decided with the maintainer on 2026-11-09, in the quest audit: ✅ trim it, in m0 since it +is CI cost. Build speed is outside rust2's goal as the maintainer restated it, any Rust +crate callable from any language, so the track keeps only what costs nothing. Rejected: +keeping the nightly schedule and fixing the nix job; removing the track. + +- The nightly `bench.yml` costs about 3.5 runner-hours on inputs that are all pinned (the + moq revision, the nightly, ra-jit), and a pin change already runs it on its PR. +- `Bench (moq nix)` has failed every scheduled run since 2026-11-02, and the choice it + informed was made on 2026-10-26 (`bench/build/README.md`). +- The `pure` corpora were built for the subset compiler, closed on 2026-11-06 + ([m4](/quest/m4/README.md#plan)). +- What stays: the four quests that wait on upstream + ([Cranelift](/quest/m3/cranelift-unwind.md), [relink](/quest/m3/relink.md), + [TPDE](/quest/m4/tpde.md), [the fork ledger](/quest/m4/fork.md)), the benchmark they + measure with, and rust2-lint, which moq's CI plans to adopt. + +Public API: none. diff --git a/quest/m0/checks/README.md b/quest/m0/checks/README.md index 474272ec7..e5120c6eb 100644 --- a/quest/m0/checks/README.md +++ b/quest/m0/checks/README.md @@ -11,7 +11,8 @@ change touches, builds warm, and runs the differential fixtures optimized, while Decided with the maintainer on 2026-11-01, from an audit of the gate's and agents' checks (nixos2 and the host's systemd accounting): copy how moq works. `just ci` before every push -stays the rule, and costs less once it selects by the diff; CONTRIBUTING.md and the skills +stays the rule, and costs less now that it selects by the diff +([#421](https://github.com/moq-dev/rust2/pull/421)); CONTRIBUTING.md and the skills keep their rules. All of it is m0, since every agent pays the cost now. Each child reports its CPU saved, cold and warm, against `origin/main`. diff --git a/quest/m0/delete-uniffi.md b/quest/m0/delete-uniffi.md new file mode 100644 index 000000000..b8809f493 --- /dev/null +++ b/quest/m0/delete-uniffi.md @@ -0,0 +1,40 @@ +# [M] Delete the UniFFI fork + +## Goal + +`uniffi/` and everything that exists only for it are gone: the root Cargo workspace, which +holds only the fork (`sh/dispatch.sh`), the flake's `uniffi*` packages and `.#uniffi` +shell, the nightly `uniffi-*` jobs, and the [binding benchmark](/bench/ffi/README.md)'s +harness, which builds moq-ffi against the fork (`run.sh`, its `fork`, `control`, and `base` +variants, the ubrn browser runner, `pins.toml`, and `bench-ffi.yml`). The benchmark's +README keeps what was measured: the BoltFFI comparison and the fork's generated sizes. +`just ci` and the nightly report their cost before and after. + +## Plan + +Decided with the maintainer on 2026-11-09, in the quest audit: ✅ delete it now. moq decided +on 2026-11-02 never to build on it ([m1](/quest/m1/README.md#plan)), so its consumers here +were only the benchmark's baseline and CI: 1,426 of the repository's 3,596 tracked files +(118k lines), about 7 minutes of each nightly gate, and nightly jobs of 46 minutes +(fixtures), 11 (install), 14 (C and C++), and 100 to 125 (the benchmark on Linux and +macOS). `UniFFI fixtures` has failed every scheduled run since 2026-11-02, and the +benchmark's Swift job since then too. Rejected: keeping it until moq switches, with a +retirement quest; keeping it as is. + +- The baseline no longer matched what moq ships anyway: moq's generator pins moved on + (go `.2`, dart `.7`, cpp `.7`) from the ones `uniffi/README.md` lists. +- Docs the deletion makes stale change with it: the root `README.md`'s fork bullet, + nix command, and license note, `AGENTS.md`'s first line ("to any language through a fork + of UniFFI"), `quest/README.md`'s Plan, and `ffi/README.md`'s first paragraph. +- Rename [the Swift-on-arm wait](/quest/m2/uniffi-swift-arm.md) with `quest mv` to a name + without `uniffi`, and rewrite its Goal around rust2-ffi's skips, which are all it then + holds; `ffi/tests/fixtures.rs`, `ffi/justfile`, and `ffi/README.md` cite its path. +- The exporter's own numbers come from + [its benchmark quest](/quest/m1/ffi-export/bench.md), which rebuilds the harness on + rust2-ffi. +- moq's `quest/m3/ffi-exporter.md` links quests here that are long deleted + (`quest/m1/uniffi.md`, `ffi-bytes.md`, `kt-ffi-calls.md`, `ffi-poll-protocol.md`) and + carries fork-era requirements; report them in the PR for moq's tree to fix. + +Public API: the flake's `uniffi*` packages and `.#uniffi` shell are removed (breaking for +anyone pinning them; no consumer does). diff --git a/quest/m0/nightly-green.md b/quest/m0/nightly-green.md new file mode 100644 index 000000000..e686734e8 --- /dev/null +++ b/quest/m0/nightly-green.md @@ -0,0 +1,43 @@ +# [M] The nightly passes + +## Goal + +Every job of the scheduled `nightly.yml` run passes on `main`, each failure fixed at its +cause, and a regression in a suite only the nightly runs cannot go unseen for a week again. + +## Plan + +Found in the 2026-11-09 quest audit: the nightly has failed every scheduled run since +2026-10-29, with no quest owning it. Decided with the maintainer on 2026-11-09: ✅ m0, +after [the fork's deletion](/quest/m0/delete-uniffi.md), which removes the `uniffi-*` +jobs, so this fixes only what remains. Rejected: m1. + +What failed on 2026-11-09 (run +[42655190735](https://github.com/moq-dev/rust2/actions/runs/42655190735), at `f7a0c279`): + +- `Gate` and `TypeScript package`: rust2-ts's `bindgen` fixture differs from native in 1 + of 5073 cases, a NaN's payload through the variadic import (`max_of` in + `ts/fixtures/bindgen/src/harness.rs`: native `18446472823932715226`, translated + `9221120237041090560`). "Encoder in Chromium" runs after that step, so the WebCodecs + encoder proof [#681](https://github.com/moq-dev/rust2/pull/681) added has never run in + CI. Reproduce first; the fix is in the translation or the import's glue, not in making + the harness blind to NaN payloads, unless JS cannot keep one through a call, which the + PR then shows. +- "Export to Kotlin on Android" at API 35: `the GC frees lent bytes nothing reaches`, for + at least three nights. +- `just ffi moq` likely fails on `main` since + [#668](https://github.com/moq-dev/rust2/pull/668): the exporter now refuses every + whole-crate `pub use` (`ffi/src/collect.rs`), and `ffi/moq/ffi.toml` excludes nine of + the eleven at the pin but not hang's and moq-tokio's `pub use moq_net;`. Not yet run in + CI, since no nightly ran after the merge. [The pin bump](/quest/m1/ffi-export/refusals/moq-pin.md) + removes them if it lands first. + +Open: what keeps it green. Recommended: the nightly's failure opens or updates one GitHub +issue the maintainer sees, and `just ffi moq`'s export step joins `just ci` when `ffi/` +or the pin changes, if it fits the gate's budget; measure it and report. + +Public API: none. + +## Required + +- [Delete the UniFFI fork](/quest/m0/delete-uniffi.md) - the nightly's `uniffi-*` jobs, one of them red, are deleted rather than fixed diff --git a/quest/m1/README.md b/quest/m1/README.md index c92c3d4d0..31c2efcd4 100644 --- a/quest/m1/README.md +++ b/quest/m1/README.md @@ -2,20 +2,23 @@ ## Goal -The work moq is waiting on: the exporter that replaces the UniFFI fork moq +The work moq is waiting on: the exporter that replaces the UniFFI generator forks moq builds its bindings with today. ## Plan Decided with the maintainer on 2026-10-25, from the [spike](https://github.com/moq-dev/rust2/pull/41): the [exporter](/ffi/README.md), which -turns a crate's `pub` API into bindings with no attributes, replaces the UniFFI fork. The -fork stays in maintenance and keeps building moq's bindings until then; its feature quests -are folded into the [exporter's](/quest/m1/ffi-export/README.md). The -[binding benchmark](/bench/ffi/README.md) continues as the exporter's baseline. +turns a crate's `pub` API into bindings with no attributes, replaces the UniFFI fork, +whose feature quests are folded into the [exporter's](/quest/m1/ffi-export/README.md). Decided 2026-11-02 (maintainer, moq's /quest-unblock): the fork is not published to crates.io. moq skips it and switches straight to the exporter ([moq's switch](https://github.com/moq-dev/moq/blob/main/quest/m3/ffi-exporter.md)), so publishing would only commit permanent `rust2-*` names to code already slated for retirement. moq keeps its own pinned generator forks until the exporter is ready. +Decided with the maintainer on 2026-11-09, in the quest audit: ✅ the fork is deleted now +([its quest](/quest/m0/delete-uniffi.md)), since moq never builds on it and its only +consumers here were the binding benchmark's baseline and nightly jobs. Rejected: keeping it +until moq switches; keeping it as is. + ## Required -- [Exporter](/quest/m1/ffi-export/README.md) - moq's bindings exported from its crates' `pub` API, replacing the fork +- [Exporter](/quest/m1/ffi-export/README.md) - moq can pin an exporter that exports its crates' `pub` API in every language it ships diff --git a/quest/m1/ffi-export/README.md b/quest/m1/ffi-export/README.md index 6b68fe4b9..9b3c46b5f 100644 --- a/quest/m1/ffi-export/README.md +++ b/quest/m1/ffi-export/README.md @@ -2,11 +2,12 @@ ## Goal -[rust2-ffi](/ffi/README.md) replaces [the UniFFI fork](/uniffi/README.md) for moq: it exports -what moq's bindings need from moq's own crates, with no attributes and no hand-written wrapper -beyond what the census shows is genuinely new API, and generates the languages moq ships on its -C ABI. moq-ffi shrinks to that remainder. The fork stays in maintenance, building moq's bindings -until then. +moq can pin [rust2-ffi](/ffi/README.md) and switch its bindings to it: the exporter exports +what moq's bindings need from moq's own crates, with no attributes, no config, and no +hand-written wrapper beyond what the census shows is genuinely new API, in every language +moq ships. It ends once every child has merged, the flake has an `ffi` package, and the +root `README.md` lists the generators, which is what +[moq's wait](https://github.com/moq-dev/moq/blob/main/quest/m3/rust2-exporter.md) checks. ## Plan @@ -15,41 +16,68 @@ Decided with the maintainer on 2026-10-25, from the spike the fork's paused feature quests into it. Decided with the maintainer on 2026-10-27: this is an epic, with each item a child ranked by -what the census refuses and what moq waits on first, then the runtime, then the generators. -What stays here is the end-to-end: moq-ffi shrinks to its remainder and takes the stand-in -package's place. +what the census refuses and what moq waits on first. -Every generator: +Decided with the maintainer on 2026-11-09, in the quest audit: + +- ✅ The epic ends where moq can pin it. Rejected: ending with moq-ffi shrunk to its + remainder, which only [moq's switch](https://github.com/moq-dev/moq/blob/main/quest/m3/ffi-exporter.md) + does, and that waits on this epic, so each waited on the other; ending at a clean census + with the children below as a later epic, which would break the generated APIs after moq + adopts them. Swapping the stand-in package for the real moq-ffi is a pin bump after + moq's switch. +- ✅ [Sessions on sans-IO QUIC](/quest/m3/session-quic.md) left the epic for m3, since it + waits on moq's m3 driver and would hold the epic open. Rejected: m2; staying in m1. +- ✅ A binding's types are lossless, with checked conversions to and from the idiomatic + ones ([lossless](/quest/m1/ffi-export/lossless.md)). Rejected: idiomatic types whose + truncations and aborts become errors; today's mappings. +- ✅ A failed run-time check raises the language's own error, and a Rust panic aborts + ([checks-raise](/quest/m1/ffi-export/checks-raise.md)). Rejected: catching panics at the + boundary; aborting on every failed check. +- ✅ A type or future that is not `Send` exports, checked by thread + ([not-send](/quest/m1/ffi-export/not-send.md)). Rejected: a Rust-owned thread for every + such object; keeping the refusal. +- ✅ No opt-out: `#[doc(hidden)]` items stay left out, as they are no part of a crate's API, + but no quest or PR recommends the attribute to get an item past the exporter, and moq + bans it ([moq's doc-hidden quest](https://github.com/moq-dev/moq/blob/main/quest/m2/api-shape/doc-hidden.md)). + An item C cannot take exports by a new rule, or is refused and its signature changes. + Rejected: `#[doc(hidden)]` as the opt-out; exporting hidden items too. + +The Swift, Python, Go, and Dart generators landed in +[#406](https://github.com/moq-dev/rust2/pull/406), [#407](https://github.com/moq-dev/rust2/pull/407), +[#411](https://github.com/moq-dev/rust2/pull/411), and [#422](https://github.com/moq-dev/rust2/pull/422), +each generating the crate's own API in its language's idiom, as Kotlin does (decided with +the maintainer on 2026-11-01). Only Python skips the C ABI, since a CPython extension calls +about 40 times faster than `ctypes`; the others call C natively. + +Every generator, the [targets m3 plans](/quest/m3/README.md) included: - checks `_contract()` against the header's contract when it loads, before any - other call, and fails loud on a mismatch; -- is measured on the [binding benchmark](/bench/ffi/README.md); -- keeps the C ABI's bulk and zero-copy payloads, with no per-byte loop on either side; -- reports its generated size, raw and gzipped, against the fork's (Python 653 KiB, Kotlin - 914 KiB, Swift 684 KiB for moq-ffi); -- wraps the C ABI's futures in the language's native future. Those futures already take the - result in the poll that finishes them, so a cancel never loses a frame, cost three crossings - when ready at once, and reuse a repeated waker. The native future must latch a wake that - lands between `PENDING` and parking, prove no loss on cancel in a test that forces that - window, and pick a semantic for concurrent pollers. A hot read still makes a future handle - per read; measure it before adding a per-handle poll. Decided with the maintainer on - 2026-10-27: async lands with each generator, not as its own quest. -- drives each of moq's `Driver`s from the language's own event loop. Decided with the - maintainer on 2026-10-27: the C ABI exports the poll, with no generated thread and no - moq-tokio helpers, which would tie bindings to tokio. A binding holds one `kio::Waiter` - per driver, made from its waker, since what a poll registers lives only as long as its - waiter; it polls on each wake and at the deadline the poll returns, and frees the driver - to cancel it. Decided with the maintainer on 2026-10-27, for Go: a cancel ends only that - read and leaves the stream usable, as in Rust, since a cancelled C ABI future loses - nothing; moq's `go/wrapper/doc.go`, which ends the stream, changes with the generator. - Each driver runs on a goroutine that lives as long as its object, and a call's `ctx` - bounds only that call. + other call, and fails loud on a mismatch (done in all six); +- keeps the C ABI's bulk and zero-copy payloads, with no per-byte loop on either side (done); +- wraps the C ABI's futures in the language's native future (done). Those futures already + take the result in the poll that finishes them, so a cancel never loses a frame, cost three + crossings when ready at once, and reuse a repeated waker. The native future must latch a + wake that lands between `PENDING` and parking, prove no loss on cancel in a test that + forces that window, and pick a semantic for concurrent pollers. Decided with the + maintainer on 2026-10-27: async lands with each generator, not as its own quest. +- drives each of moq's `Driver`s from the language's own event loop (done in all seven + `ffi/moq/moq.*`). Decided with the maintainer on 2026-10-27: the C ABI exports the poll, + with no generated thread and no moq-tokio helpers, which would tie bindings to tokio. A + binding holds one `kio::Waiter` per driver, made from its waker, since what a poll + registers lives only as long as its waiter; it polls on each wake and at the deadline the + poll returns, and frees the driver to cancel it. Decided with the maintainer on + 2026-10-27, for Go: a cancel ends only that read and leaves the stream usable, as in + Rust, since a cancelled C ABI future loses nothing; moq's `go/wrapper/doc.go`, which ends + the stream, changes with the generator. Each driver runs on a goroutine that lives as + long as its object, and a call's `ctx` bounds only that call. +- is measured, with its generated size raw and gzipped + ([bench](/quest/m1/ffi-export/bench.md) owns both, and the per-handle poll a hot read + may need). -Decided with the maintainer on 2026-11-01: the Swift, Python, Go, and Dart generators are -independent and unranked, and each generates the crate's own API in its language's idiom, as -Kotlin does. Only Python skips the C ABI, since a CPython extension calls about 40 times faster -than `ctypes`; the others call C natively. moq's packages switch to them in moq's own -[quest](https://github.com/moq-dev/moq/blob/main/quest/m3/ffi-exporter.md). +What stays here once every child has merged: the flake's `ffi` package, the root +`README.md` listing the generators, and the generator rules above moved into +`ffi/README.md`, since this file is deleted. Decided with the maintainer on 2026-11-02 ([moq-dev/rust2#460](https://github.com/moq-dev/rust2/pull/460)): the RSS glibc's per-thread @@ -62,4 +90,10 @@ names and defaults, and fails the export on a public item it cannot export. ## Required - [Every public item exports](/quest/m1/ffi-export/refusals/README.md) - the census lists no refusals and `ffi.toml` is deleted, so the exporter reads no config -- [Sessions on sans-IO QUIC](/quest/m1/ffi-export/session-quic.md) - bindings move UDP datagrams and drive the session's poll themselves, over moq's runtime-free QUIC driver once it lands +- [Export for a target](/quest/m1/ffi-export/target.md) - `--target` exports the API that target compiles, so `cfg`-gated items are right on Android, iOS, Windows, and wasm +- [Lossless binding types](/quest/m1/ffi-export/lossless.md) - durations, addresses, paths, text, and integers cross exactly, with checked conversions to the idiomatic types +- [Failed checks raise](/quest/m1/ffi-export/checks-raise.md) - a failed borrow, move, or close check is the language's own error from one accounting, and only a panic aborts +- [Types that are not Send export](/quest/m1/ffi-export/not-send.md) - a `!Send` object or future is checked by thread, with an owner thread where a language has none +- [No name clash fails an export](/quest/m1/ffi-export/names.md) - every generator names every item by rule, so no error tells a crate to rename or exclude one +- [The glue builds what the export checked](/quest/m1/ffi-export/reproducible.md) - the generated crate carries its lockfile, and a discriminant or nightly-only item it cannot build is refused at export +- [The exporter on the binding benchmark](/quest/m1/ffi-export/bench.md) - committed call, throughput, and size numbers per language against the Rust floor, whose report plans the speed work diff --git a/quest/m1/ffi-export/bench.md b/quest/m1/ffi-export/bench.md new file mode 100644 index 000000000..6698ea87d --- /dev/null +++ b/quest/m1/ffi-export/bench.md @@ -0,0 +1,46 @@ +# [M] The exporter on the binding benchmark + +## Goal + +The [binding benchmark](/bench/ffi/README.md) measures [rust2-ffi](/ffi/README.md)'s +bindings and commits the numbers: per language, the calls and throughput cases of its +BoltFFI table against a Rust floor, and each generated binding's size, raw and gzipped, +for the moq stand-in. Its report ranks what a call spends, and plans the speed work the +numbers justify as quests. + +## Plan + +Found in the 2026-11-09 quest audit: no exporter measurement is committed. The benchmark +measures moq-ffi over the UniFFI fork, `just ffi bench` (`ffi/justfile`, +`ffi/moq/bench/export.*`) prints timings and records nothing, and the epic's rule that +every generator "is measured on the binding benchmark" and "reports its generated size" +had no owner. Speed ranks above ergonomics here (AGENTS.md), so it needs a baseline. + +- The targets: BoltFFI's cells in `bench/ffi/README.md` (Kotlin `noop()` 5.6 ns, a record + return 61 ns; Python `noop()` 21.6 ns), and UniFFI's unreleased JNI bindgen, which its + maintainer measured at 7.7 ns a no-op against BoltFFI's 8.2 + ([gist](https://gist.github.com/bendk/5054d1477361125ef430e2ccabfd5aed), from + [mozilla/uniffi-rs#2911](https://github.com/mozilla/uniffi-rs/pull/2911)). The fork's + sizes for moq-ffi: Python 653 KiB, Kotlin 914 KiB, Swift 684 KiB. +- The structural costs the audit read off the code, for the report to confirm or drop: + - an owned `String`, `Vec`, or `Bytes` argument is always copied at the C ABI + (`glue.rs`), and again in Kotlin and Dart, on moq's frame-write path; + - Kotlin crosses records, `Duration`, and `SocketAddr` through a byte-array codec, about + four JVM allocations and three copies an argument, resolves JNI symbols by name, and + takes a global lock on each handle's creation and free (`kotlin.rs`, `rt.kt`, `jni.rs`); + - a future costs three heap allocations and three crossings when ready at once + (`rt.rs`, `glue.rs`), and a hot read makes one per read: measure it before adding a + per-handle poll; + - each returned buffer is a `Box` and a free crossing, each returned object an + `Arc::new`, and each object lent to a closure an `Arc::new` per call (`rt.rs`); + - Dart hands each call to a worker thread under a global mutex while a relay is held + (`ffi/runtime/dart.rs`), and Go takes a global mutex on lent-buffer frees (`rt.go`). + The doubled borrow accounting is [checks-raise](/quest/m1/ffi-export/checks-raise.md)'s. +- No threshold is set, as in the other benchmarks: the numbers inform the quests the + report plans. + +Public API: none. + +## Required + +- [Delete the UniFFI fork](/quest/m0/delete-uniffi.md) - the benchmark's harness stops building the fork, so this rebuilds it on the exporter diff --git a/quest/m1/ffi-export/checks-raise.md b/quest/m1/ffi-export/checks-raise.md new file mode 100644 index 000000000..821a47d39 --- /dev/null +++ b/quest/m1/ffi-export/checks-raise.md @@ -0,0 +1,40 @@ +# [L] Failed checks raise + +## Goal + +A run-time check a [rust2-ffi](/ffi/README.md) binding fails, for a rule its language +cannot express, raises the language's own error and leaves the process running: a borrow +Rust would refuse (a `&mut self` call while another call holds the handle), a call on a +moved-out or closed handle, or an object lent for a call used past it. Kotlin, Python, Dart, +and Swift throw, and Go returns an `error`. C and C++ built without exceptions +still abort with the message, and a Rust panic aborts everywhere. Each handle's borrows +are counted once per call, not in the wrapper and again in the runtime. + +## Plan + +Decided with the maintainer on 2026-11-09, in the quest audit: ✅ checks raise and panics +abort. Rejected: raising panics too, which costs unwind tables and landing pads in every +export and a poisoned state for an object that panicked mid-call; aborting on every failed +check, as today. The check runs before any Rust code does, so nothing is corrupted when it +fails. AGENTS.md carries the rule. + +- Today `ffi/runtime/rt.rs` aborts through `fail` at 26 sites, reachable from safe + multithreaded Kotlin, Go, and Python, while the wrappers already raise for a closed + handle (`rt.kt`, `rt.swift`, `rt.go`). Kotlin, Swift, and Go keep their own hold count + beside the runtime's borrow flag, so a call costs four atomic operations per handle + where C's costs two. +- One accounting, where it can raise: the binding's. Measure a call before and after on + [the benchmark](/quest/m1/ffi-export/bench.md); the target is the C ABI's two atomics + per handle or fewer. +- Swift's `async throws` and throwing functions already exist for `Result`; a call that + can fail only a check becomes throwing too, which changes its signature. Go already + returns `rt.ErrClosed` from every call that takes a handle. +- A foreign exception inside a callback or lent closure still aborts, since Rust has + nowhere to unwind it to. + +Public API: which generated functions throw or return an error (breaking in Swift), and +the error types the runtimes add. + +## Required + +- [The exporter on the binding benchmark](/quest/m1/ffi-export/bench.md) - the committed baseline this change's call cost is measured against diff --git a/quest/m1/ffi-export/lossless.md b/quest/m1/ffi-export/lossless.md new file mode 100644 index 000000000..c4274d177 --- /dev/null +++ b/quest/m1/ffi-export/lossless.md @@ -0,0 +1,53 @@ +# [L] Lossless binding types + +## Goal + +Every type a [rust2-ffi](/ffi/README.md) binding takes or returns holds every value of its +Rust type exactly: a value passed in and returned is the same value, and no conversion +truncates, clamps, panics, traps, or aborts. A native type stands in only where it is +exact; otherwise the runtime's own type crosses, with checked conversions to and from the +idiomatic one that fail with the language's error. Proven in every language's `api` test +by the edge values of each type crossing both ways. + +## Plan + +Decided with the maintainer on 2026-11-09, in the quest audit: ✅ lossless always, with +helpers that convert to and from the idiomatic type, checked. Rejected: idiomatic types +whose silent losses become errors, which still fail at run time on values that do not fit; +today's mappings. AGENTS.md carries the rule. + +What the audit found, each a place native Rust would have gone on: + +- `Duration`: Python's `timedelta` and Dart's `Duration` truncate to the microsecond and + clamp (`ffi/runtime/python.rs`, `rt.dart`); Go's `time.Duration` panics on a negative + one or past 292 years (`ffi_cgo.go`); Kotlin's maps past `i64` seconds to `INFINITE` + and back to `i64::MAX` seconds (`rt.kt`). Seconds and nanoseconds, as std holds them, + cross exactly. moq's switch also asks that Go stop panicking on a negative one and that + a duration field default to zero + ([moq's quest](https://github.com/moq-dev/moq/blob/main/quest/m3/ffi-exporter.md)). +- Socket addresses: Kotlin throws and Go panics on an IPv6 flow label, and Go on a zone it + cannot look up; `SocketAddrV4` given an IPv6 address aborts in C and C++, panics in Go, + and throws in Kotlin and Dart. The runtime's own `SocketAddr`, as Swift and Dart have, + and a C struct per family, hold each exactly. +- Paths: one that is not UTF-8 throws in Kotlin, traps in Swift, and aborts in Dart; Go + and Python already keep its bytes. +- Text: a Go `string` or C++ view that is not UTF-8 aborts the process (`rt.rs`), and an + invalid `char` aborts from Kotlin's `Int`, Go's `rune`, and C++'s `char32_t`. A value + the language's own type allows and Rust's does not is checked in the binding, which + raises. +- Integers: a Dart `u64` past `i64` reads negative, and Kotlin's `usize` crosses as `u64` + and is cast, truncating on a 32-bit ABI. +- `Hash`: Dart's `hashCode` throws on a `PartialEq` type and Kotlin's returns 0; the IR + carries no `Hash`. Export it where the type implements it. + +Each mapping reports its cost on [the benchmark](/quest/m1/ffi-export/bench.md); none +should add a crossing or an allocation, and several remove one. Where a language's range +is the limit and no exact type exists at a call's speed, such as a Python `int` past +`u64` raising `OverflowError`, the raise stays: it rejects a value Rust's type cannot hold. + +Public API: every generated API's types for durations, addresses, paths, and text +(breaking), and the runtimes' conversion helpers. + +## Required + +- [Failed checks raise](/quest/m1/ffi-export/checks-raise.md) - how a binding raises, which the checked conversions and invalid text use diff --git a/quest/m1/ffi-export/names.md b/quest/m1/ffi-export/names.md new file mode 100644 index 000000000..ef6a578ea --- /dev/null +++ b/quest/m1/ffi-export/names.md @@ -0,0 +1,34 @@ +# [M] No name clash fails an export + +## Goal + +Every [rust2-ffi](/ffi/README.md) generator names every exported item by a fixed rule +that cannot collide, so no export fails telling the crate's author to "rename one in the +source or exclude it in the config". A fixture whose items clash in each language today +(two types whose segments differ only by case, a member named as one the language gives +its type, a Dart name the runtime declares) exports and is called from each. + +## Plan + +Found in the 2026-11-09 quest audit: these errors are raised by the generators, outside +the census, so "the census lists no refusals" does not cover them, and once +[the refusals epic](/quest/m1/ffi-export/refusals/README.md) deletes `ffi.toml` their +advice names a config that no longer exists. The sites: `kotlin.rs` (two types whose +names match by case), `python.rs`, `cpp.rs`, `dart.rs`, `go.rs`, `swift.rs`, the C names' +own registry (`audio_encoder::finish` against `audio::encoder_finish`), and +`ffi/README.md`'s Dart section. AGENTS.md calls each source change a consumer needs a gap +to close. + +Every generator runs on every export, so one language's clash fails all of them: a +crate that never ships Dart cannot export because of a Dart name. + +- Extend the rule the generators already share for two types whose names match: a `_` + where their paths split, one more segment at a time until they differ, and a trailing + `_` against a name the language or runtime owns. +- A name depends on every item exported, so an upstream addition can lengthen another's + name, as generic instances' already can. Keep that rule rather than add stable mangling. +- Open: the C names, where joining segments with `_` loses where they split. Recommended: + the same one-more-segment rule with a doubled `_` at the split that differs, so + `moq_audio_encoder_finish` stays for the item whose path is shorter. + +Public API: the names of items that clash today, which fail to export now. diff --git a/quest/m1/ffi-export/not-send.md b/quest/m1/ffi-export/not-send.md new file mode 100644 index 000000000..2b8be2299 --- /dev/null +++ b/quest/m1/ffi-export/not-send.md @@ -0,0 +1,55 @@ +# [L] Types that are not Send export + +## Goal + +[rust2-ffi](/ffi/README.md) exports an object or future that is not `Send`, bound to the +thread that made it: a call, poll, or drop from another thread never runs Rust code on +that value there. Three of moq's own are refused for it at moq `ffcb7cd9`: `clock::Entered`, +`clock::Rng`, and `catalog::Guard` +([#675](https://github.com/moq-dev/rust2/pull/675)). Proven per language by a fixture +type holding an `Rc`, called, awaited, and dropped from a second thread. + +## Plan + +Decided with the maintainer on 2026-11-09, in the quest audit: ✅ a thread check, with a +shim only where a language has no stable thread. Rejected: a Rust-owned thread for every +such object, which costs two thread handoffs a call even from a single-threaded caller +such as Python's event loop; keeping the refusal, as +[small shapes](/quest/m1/ffi-export/refusals/small-shapes.md) had it. The maintainer: +an assert is fine "where a language has no concept of something being thread local". + +- The handle records its owner thread, and each call compares it: no atomics, so it costs + less than a `Sync` type's borrow. The wrong thread + [raises](/quest/m1/ffi-export/checks-raise.md). +- A free from another thread never drops the value there. GC finalizers run elsewhere: + Kotlin's phantom-reference daemon thread, Go's `runtime.AddCleanup` goroutine, Python's + collector, Swift's `deinit` on the releasing thread, and Dart's finalizer with no + current isolate. The free queues to the owner, which drains it at its next call, and a + queue must not wait on a call that never comes: where the binding has an event loop on + the owner thread (Python's loop, a C++ executor), queueing a free schedules a drain + there. Open: what drains it elsewhere, and once the owner thread has exited. + Recommended: drop the queue from the thread's exit hook (a thread-local's destructor), + and leak with a logged message only where the thread is already gone, as PyO3's + `unsendable` does. +- Where the language can say it, its compiler enforces the rule instead: a Swift object + that is `Send` but not `Sync` is not `Sendable` (today every object is + `@unchecked Sendable`, `ffi/src/swift.rs`). Confinement is not thread affinity, though: + a Swift actor's jobs run on any thread of its pool, and a Dart isolate need not keep its + OS thread, while Rust cannot tell an `Rc` from a guard pinned to its thread, as + [#675](https://github.com/moq-dev/rust2/pull/675) calls moq's three. So a `!Send` value needs the check or the owner thread below in + Swift and Dart too, unless an executor pinned to the owner thread holds it; test a call + after a suspension, and through Dart's relay. +- Where a language has no stable thread, a Rust-owned thread holds the value and runs + its calls, two handoffs each: Go, whose goroutines move between OS threads, and a + `!Send` future awaited on a pool, as Kotlin's dispatchers, Swift's executors, and Go's + goroutines poll it. The future is spawned on the owner thread and the binding polls a + `Send` handle to it. Dart's relay (`ffi/runtime/dart.rs`'s `Home`) is the same + machinery in the other direction. +- wasm32 has one thread, so [rust2-wasm](/quest/m2/wasm/README.md) exports these with no + check. + +Public API: the items that now export, and Swift's `Sendable` conformances (breaking). + +## Required + +- [Failed checks raise](/quest/m1/ffi-export/checks-raise.md) - the wrong-thread check raises through the same accounting diff --git a/quest/m1/ffi-export/refusals/README.md b/quest/m1/ffi-export/refusals/README.md index b1319240b..69a2a336e 100644 --- a/quest/m1/ffi-export/refusals/README.md +++ b/quest/m1/ffi-export/refusals/README.md @@ -2,15 +2,21 @@ ## Goal -[rust2-ffi](/ffi/README.md) exports every `pub` item of the moq facade it is given, so -`just ffi census` lists no refusals at the moq pin and `ffi.toml` is deleted: the exporter -reads no config, and an item it cannot export fails the export with a span. +[rust2-ffi](/ffi/README.md) exports every `pub` item of the moq facade it is given, or +leaves out and lists in the census what C can never hold, so `just ffi census` lists no +refusals at the moq pin and `ffi.toml` is deleted: the exporter reads no config, and an +item it cannot export fails the export with a span. What stays here once every child has merged: `ffi/moq/ffi.toml`, the `--config` flag, `RUST2_FFI_CONFIG`, and `config.rs`'s `Config` and `exclude` are deleted, with the fixtures' `ffi/fixtures/api/ffi.toml` and `ffi/fixtures/reject/ffi.toml` and the exclude tests in `ffi/tests/fixtures.rs`; `ffi/README.md` and the recipes stop naming them. -`config.rs`'s `Package` and environment constants stay. +`config.rs`'s `Package` and environment constants stay. Also deleted: the `--config` +parsing rust2-ffi takes from `driver/src/cargo.rs`, which already has `parse_configless`; +`go/bench/ffi.toml` and the `--config` in `go/justfile`'s bench recipe, whose `package` +and `prefix` keys `config.rs` already rejects; and "exclude it in the config" in +`ffi/src/driver.rs`'s refusal notes and `ffi/fixtures/reject/expected.txt`. The generators' +own clash errors are [names](/quest/m1/ffi-export/names.md)'s. ## Plan @@ -33,18 +39,21 @@ Decided with the maintainer on 2026-11-07, from - ✅ The stand-in re-exports only the session side moq-ffi ships from moq-tokio. Rejected: all of moq-tokio, whose tokio QUIC driver, CLI helpers, worker groups, and listeners no binding calls; the sans-IO driver bindings use instead is - [session-quic](/quest/m1/ffi-export/session-quic.md). Children leave the moq-tokio items + [session-quic](/quest/m3/session-quic.md). Children leave the moq-tokio items outside that session side alone, since narrowing the stand-in drops them. -- ✅ Narrowing the stand-in is the first child, so the others count only what stays - (decided in the quest audit). Rejected: narrowing last, as this epic's own work. +- Narrowing the stand-in was to land first; the 2026-11-09 audit dropped the order, as + [its quest](/quest/m1/ffi-export/refusals/narrow-standin.md) records. - ✅ What C can never hold, an object no export returns or lends to a closure, or a generic type no export instantiates, is left out of the bindings and kept listed in the census. Rejected: refusing them, which would make moq add constructors it has no use for. - ✅ This epic, with the children below ranked by what moq-ffi's hand-written wrapper still covers. -The census at moq `ffcb7cd9` (2026-11-07) refuses 1111 items, which `ffi/moq/ffi.toml`'s -300 excludes hide by naming whole modules. About 790 lie in crates moq re-exports whole. +The census at moq `ffcb7cd9` (2026-11-07) refused 1111 items, which `ffi/moq/ffi.toml`'s +347 excludes hide by naming whole modules. About 790 lay in crates moq re-exports whole, +which [#668](https://github.com/moq-dev/rust2/pull/668) has since made one refusal each, +and [#671](https://github.com/moq-dev/rust2/pull/671) moved what C can never hold out of +the refusals, so the figures below predate both. The other 324 are moq's own, 107 of them in moq-tokio (61 under `quic`); each child gives its construct's count, and an item counts under every construct it hits. The counts include the moq-tokio items narrowing drops; each child measures again once it lands. 17 more refuse diff --git a/quest/m1/ffi-export/refusals/moq-pin.md b/quest/m1/ffi-export/refusals/moq-pin.md index 93271d7eb..0e97e6500 100644 --- a/quest/m1/ffi-export/refusals/moq-pin.md +++ b/quest/m1/ffi-export/refusals/moq-pin.md @@ -1,11 +1,12 @@ -# [XS] Bump the moq pin past its re-export and reachability fixes +# [S] Bump the moq pin past its re-export and reachability fixes ## Goal The moq pin moves to a revision whose crates re-export no whole crate, moq's own included, and where every type a facade export names is reachable from the facade's root. `just ffi census` there lists no whole-crate re-export refusal and no "type not reachable from the -crate root" refusal of moq's own (18 at moq `ffcb7cd9`). +crate root" refusal of moq's own (20 at moq `ffcb7cd9`, and 9 at moq's head on +2026-11-09, measured in [#676](https://github.com/moq-dev/rust2/pull/676)). ## Plan @@ -32,15 +33,27 @@ session side also drops `quic::driver`. Decided with the maintainer on 2026-11-07, in the quest audit: one pin bump for both, since each is an XS bump of the same pin line. Rejected: one bump per moq quest. +Both of moq's fixes merged on 2026-11-09: +[moq-dev/moq#8247](https://github.com/moq-dev/moq/pull/8247) dropped every whole-crate +re-export and re-exports `Guard`, and +[moq-dev/moq#8302](https://github.com/moq-dev/moq/pull/8302) removed +`stream::Id::{new, initiator}`. At moq's head the 9 unreachable items left are +`kio::Pending` (from `dial_tcp` and `dial_track`, reachable before only through +`pub use kio;`), `lite::version::Version`, which was already unreachable at the pin, and +moq-tokio's `quic::driver` items, which the narrowed stand-in drops. + Then bump the pin in `ts/fixtures/model/Cargo.toml`, which `ts/justfile`'s `moq_rev` reads, -and report the census before and after. +and report the census before and after. The last bump +([#390](https://github.com/moq-dev/rust2/pull/390)) also changed `ffi/moq/ffi.toml`, every +`ffi/moq/moq.*`, and rust2-ts's model fixture, bench, and expected output, which share the +pin: run `just ts census` too, and update the count in the +[model epic](/quest/m2/ts-model/README.md). ## Waiting -- moq-dev/moq: [quest/m2/api-shape/reexports.md](https://github.com/moq-dev/moq/blob/main/quest/m2/api-shape/reexports.md), - planned in [moq-dev/moq#8233](https://github.com/moq-dev/moq/pull/8233), drops every - whole-crate re-export, moq's own included, and re-exports `Guard` from `container`. Done - once that quest is gone from moq's `main` and the census at its head lists no whole-crate - re-export. -- moq-dev/moq: [quest/m2/moq-quic-unused-surface.md](https://github.com/moq-dev/moq/blob/main/quest/m2/moq-quic-unused-surface.md) - removes `stream::Id::{new, initiator}`. Done once that quest is gone from moq's `main`. +- Maintainer: whether the bump lands before moq fixes the two unreachable types, narrowing + this Goal to what the census then lists. [#676](https://github.com/moq-dev/rust2/pull/676) + recommends landing it, since the sibling quests need moq's head. +- moq-dev/moq: `kio::Pending` and `lite::version::Version` are reachable from moq-net's + root, by name or by wrapping. No moq quest owns it yet. Done when `just ffi census` at + moq's head lists no "type not reachable from the crate root" item for them. diff --git a/quest/m1/ffi-export/refusals/narrow-standin.md b/quest/m1/ffi-export/refusals/narrow-standin.md index 61dbd57cf..239d69626 100644 --- a/quest/m1/ffi-export/refusals/narrow-standin.md +++ b/quest/m1/ffi-export/refusals/narrow-standin.md @@ -15,13 +15,32 @@ Decided with the maintainer on 2026-11-07, from [moq-dev/rust2#655](https://github.com/moq-dev/rust2/pull/655): ✅ the stand-in re-exports only that session side. Rejected: all of moq-tokio, whose tokio QUIC driver, CLI helpers, worker groups, and listeners no binding calls; bindings drive the sans-IO driver of -[session-quic](/quest/m1/ffi-export/session-quic.md) instead. +[session-quic](/quest/m3/session-quic.md) instead. -Decided with the maintainer on 2026-11-07, in the quest audit: ✅ this lands first, so the -other children count only what stays. Narrowing drops moq-tokio's `rustls`, `notify`, +Decided with the maintainer on 2026-11-09, in the quest audit: ✅ no landing order. Each +sibling leaves the moq-tokio items outside the session side alone, and whichever PR lands +later measures the census again. Rejected: this landing first, as decided on 2026-11-07, +which no `Required` link held and which two siblings ready for review had already passed. + +The list above is not closed over what the fixtures and moq-ffi call: +[#683](https://github.com/moq-dev/rust2/pull/683) found `just ffi moq` fails with it, since +`ffi/moq/moq.*` use about 40 moq-tokio names it leaves out, such as `client::Config`, +`server::Config`, `origin::spawn`, `Backoff`, and `listen::Bind`. + +Narrowing drops moq-tokio's `rustls`, `notify`, `congestion`, `cli::usage_rs`, and `quic::web`'s `http` and `proto` re-exports, and `quic::driver`'s `Side` refusals. Every item named exists at moq `ffcb7cd9` (`rs/moq-tokio/src/lib.rs`). Update the counts in the [epic](/quest/m1/ffi-export/refusals/README.md) and its children from the new census. Public API: the stand-in's exports, which the census and the bindings' fixtures read. + +## Waiting + +- Maintainer: how the stand-in closes over what its session items name, from + [#683](https://github.com/moq-dev/rust2/pull/683), since the Goal's list alone fails + `just ffi moq`. Options: add those types and `origin::spawn` to the list, keeping the CLI + helpers and listener types out (recommended: a stand-in that cannot construct a client + is not the session side); or keep the list and rewrite the seven fixtures to it, losing + their coverage of client config, server config, and origin spawn. An agent cannot pick: + it changes which of moq-tokio's API bindings get. diff --git a/quest/m1/ffi-export/refusals/small-shapes.md b/quest/m1/ffi-export/refusals/small-shapes.md index 2aacdfd1f..13ee06737 100644 --- a/quest/m1/ffi-export/refusals/small-shapes.md +++ b/quest/m1/ffi-export/refusals/small-shapes.md @@ -4,11 +4,11 @@ The census at the moq pin refuses no item of moq's own for a small shape: 26 items at moq `ffcb7cd9` use `u128` (`Timestamp::as_nanos`), an array, a const generic, unit outside a -return, `Result` outside a return, `!`, an enum without variants, or an object or future -that is not `Send`. +return, `Result` outside a return, `!`, or an enum without variants. The count includes +the objects and futures that are not `Send`, which +[not-send](/quest/m1/ffi-export/not-send.md) owns since 2026-11-09. ## Plan -Each is its own small mapping; what has no C shape, such as an object that is not `Send` -under the C ABI's any-thread calls, keeps its refusal and goes to the maintainer as moq's -signature to change. +Each is its own small mapping; what has no C shape keeps its refusal and goes to the +maintainer as moq's signature to change. diff --git a/quest/m1/ffi-export/reproducible.md b/quest/m1/ffi-export/reproducible.md new file mode 100644 index 000000000..daec21216 --- /dev/null +++ b/quest/m1/ffi-export/reproducible.md @@ -0,0 +1,31 @@ +# [S] The glue builds what the export checked + +## Goal + +The crate `rust2-ffi --out` writes builds against exactly the dependency versions the +export checked, on the consumer's own toolchain, or the export fails with a span: no +difference between the check and the build shows up as a glue compile error or a run-time +abort. + +## Plan + +Found in the 2026-11-09 quest audit: + +- `--out` writes no `Cargo.lock`; only `ffi/justfile` copies one. A build that resolves a + newer dependency can reach the wildcard arm the glue emits for a `#[non_exhaustive]` + enum, which panics and so aborts (`ffi/src/glue.rs`). Write a lockfile beside the + manifest, as the manifest already carries its `[patch]`es: resolved for the generated + manifest, which adds dependencies of its own (`smallvec`, `pyo3-ffi`), with every + package the check resolved at the version it checked, or the export fails. The recipes + and `ffi/README.md` then build it `--locked`. +- An enum's discriminants are collected as `i128` (`ffi/src/collect.rs`) and lifted as + `i32` (`glue.rs`, `c.rs`), with no refusal for one out of range, so a wide `repr` + fails when the glue builds. Refuse it at export, at the variant's span. +- The export checks the whole graph on the pinned nightly (`ffi/rust-toolchain.toml`), + and the glue then builds on the consumer's toolchain, which the generated manifest does + not name. A build script that detects nightly, as `rustversion` and `autocfg` users do, + can give the check an item the stable build lacks. Add a fixture with such an item and + report what happens; recommended: refuse an exported item whose `cfg` depends on the + release channel, and give the generated manifest the `rust-version` the facade declares. + +Public API: the generated crate's `Cargo.lock` and `rust-version`. diff --git a/quest/m1/ffi-export/target.md b/quest/m1/ffi-export/target.md new file mode 100644 index 000000000..924920a7c --- /dev/null +++ b/quest/m1/ffi-export/target.md @@ -0,0 +1,37 @@ +# [L] Export for a target + +## Goal + +`rust2-ffi --target ` exports the `pub` API the facade has on that target, so an +item under `cfg(target_os = "android")`, `cfg(windows)`, or `cfg(target_arch = "wasm32")` +is in that target's bindings and in no other's, and the glue crate builds for the target +with no item missing or left over. Proven by a fixture whose API differs by target, +exported and built for the host and at least one other target, and by moq's census on each +target moq ships. + +## Plan + +Found in the 2026-11-09 quest audit: the check runs on the host whatever cargo's config +names (`ffi/src/cli.rs`'s `Check` sets `target: Some(host_tuple())`), and `--target` is +refused ("the export builds for the host"). So a Linux host exports moq-tokio's `unix` +module into bindings shipped to Windows, which `ffi/moq/ffi.toml` hides today with its +`tokio::unix::Error` exclude, and no host can export for wasm32, which +[rust2-wasm](/quest/m2/wasm/README.md) needs. + +Decided with the maintainer on 2026-11-09: ✅ m1, in the exporter epic, since +[the refusals epic](/quest/m1/ffi-export/refusals/README.md) deletes the excludes that +hide target-gated items. Rejected: m2 with the wasm backend, leaving the exporter host-only. + +- The check needs the target's std and, for a build script or proc macro, the host's, as + cargo already splits them; rust2-ts checks for `wasm32-unknown-unknown` through the same + front end ([`driver/`](/driver)). +- The C ABI's layouts are the target's: `usize`, pointer width, and struct layout come from + that target's check, including a 32-bit one (`ffi/runtime/jni.rs` passes `usize` as + `u64` and casts, which truncates silently on a 32-bit Android ABI). +- Open: how a binding several targets share is exported when their APIs differ, such as one + Swift package for iOS and macOS or one Kotlin file for every Android ABI. Recommended: + export once per target, and where a shared binding is asked for, refuse an item whose + export differs between its targets, at the item's span, naming both. Report first what + differs in moq at the pin between the targets it ships. + +Public API: the `--target` flag, and each target's exports. diff --git a/quest/m2/README.md b/quest/m2/README.md index 635d2f76b..4b772e7f6 100644 --- a/quest/m2/README.md +++ b/quest/m2/README.md @@ -2,9 +2,8 @@ ## Goal -Research work with a named consumer that is not waiting on it yet: the -TypeScript translator's spikes and features, and the UniFFI fork's gated or -speculative work. +Research work with a named consumer that is not waiting on it yet: rust2-wasm, moq's +browser route, and the TypeScript translator that has to beat it. ## Plan @@ -16,6 +15,10 @@ Decided with the maintainer on 2026-10-24: the translator's output ranks correct then speed, then the size of the consumer's final tree-shaken, minified JS; readability is not a goal, and types must be solid at the API boundary ([design](/ts/README.md#design)). +Decided with the maintainer on 2026-11-09, in the quest audit: WASM ships, and the +translator must beat it. [rust2-wasm](/quest/m2/wasm/README.md) is the browser route and +holds the gate; until it reports, translator work is only what the gate needs. + Decided with the maintainer on 2026-10-26: the translator targets browsers. A native runtime such as Node or Bun calls Rust through [rust2-ffi](/ffi/README.md), so Node and Bun matter only as proxies for V8 and JavaScriptCore, and a speed call that splits them @@ -26,6 +29,7 @@ The fork's feature quests folded into the [exporter](/quest/m1/ffi-export/README ## Required -- [Translate moq-net's model to TypeScript](/quest/m2/ts-model/README.md) - the model translates with kio and moq-pattern, as moq's browser route +- [rust2-wasm](/quest/m2/wasm/README.md) - a crate's `pub` API in the browser through the exporter's wasm32 target, and the gate the translator has to pass +- [Translate moq-net's model to TypeScript](/quest/m2/ts-model/README.md) - the model translates with kio and moq-pattern, for the gate to compare +- [Spike: the target language's compiler enforces Rust's rules](/quest/m2/compile-checks.md) - which of Swift's and C++'s run-time checks become compile errors, and at what cost - [Waiting: Swift 6.4 in nixpkgs-unstable](/quest/m2/uniffi-swift-arm.md) - the nightly UniFFI fixtures move back to arm -- [Waiting: ubrn reads UniFFI 0.32](/quest/m2/ubrn-release.md) - the binding benchmark's browser runner pins a ubrn release, not an open PR diff --git a/quest/m2/compile-checks.md b/quest/m2/compile-checks.md new file mode 100644 index 000000000..59c6d1800 --- /dev/null +++ b/quest/m2/compile-checks.md @@ -0,0 +1,31 @@ +# [M] Spike: the target language's compiler enforces Rust's rules + +## Goal + +A report on how many of the run-time checks [rust2-ffi](/ffi/README.md)'s Swift and C++ +bindings make become compile errors when the generators emit those languages' own +ownership forms, and what that costs their callers. It generates the stricter forms for +the `api` fixture, lists each test in `ffi/tests/api.swift` and `api.cpp` that breaks a +check today and whether the stricter binding refuses it at compile time, and shows the +fixture's script rewritten against it. + +## Plan + +Decided with the maintainer on 2026-11-09, in the quest audit: ✅ a spike in m2, whose +report decides whether the generators switch. moq's own wrapper packages sit above the +generated API, so a later break is absorbed there. Rejected: spiking it in m1 before moq +adopts the exporter; not planning it. AGENTS.md lets a binding check at run time only +what its language cannot express, so this finds where that line really is. + +- Swift: a handle as a `~Copyable` type with `consuming` methods for `self` and + `borrowing` parameters makes a call after a move a compile error, where today it traps. + The cost is a handle that is harder to store in a collection or capture in a closure. + (`Sendable` conformance by `Sync` is [not-send](/quest/m1/ffi-export/not-send.md)'s.) +- C++: `[[clang::lifetimebound]]` on a returned `view` and `[[clang::lifetime_capture_by]]` + on what a callable keeps make Clang warn where a view outlives its buffer, today C++'s + own undefined behavior; `-Wdangling` as an error is the consumer's choice. Swift + imports the same attributes through its C++ interop + ([safe interop](https://swift.org/documentation/cxx-interop/safe-interop/)). +- Kotlin, Go, Python, and Dart have no such form; say so in the report rather than look. + +Public API: none; the report scopes any later change to the generated Swift and C++. diff --git a/quest/m2/ts-model/README.md b/quest/m2/ts-model/README.md index 804e2293e..247ab04bf 100644 --- a/quest/m2/ts-model/README.md +++ b/quest/m2/ts-model/README.md @@ -8,13 +8,20 @@ calls (kio, with its clock, and moq-pattern), as it translates the codec today, ## Plan +Decided with the maintainer on 2026-11-09, in the quest audit: ✅ WASM ships, and the +translation must beat it. [rust2-wasm](/quest/m2/wasm/README.md) is moq's browser route, +and its gate compares this translation against it; moq adopts the translation only where +that report shows it faster or smaller. This supersedes the 2026-11-01 decision below. +Rejected: JS-only staying the goal; pausing this epic. + Decided with the maintainer on 2026-11-01, after moq removed its WASM build ([moq-dev/moq#7475](https://github.com/moq-dev/moq/pull/7475)): the translated model is moq's browser route, which moq's [JS-only `@moq/net`](https://github.com/moq-dev/moq/blob/main/quest/m5/rs2ts-js.md) adopts once it matches the handwritten package in a full browser workload. Once the model translates, moq measures it on the [TS benchmark](/ts/README.md#bench), consuming the driver as the `ts` package of this flake. That adds moq's `js/net/bench/frames.ts`, a group stream -read into a group, to the benchmark's workloads, which hold only its sync frame headers. The +read into a group, to the benchmark's workloads, which hold no async model workload yet +(`ts/bench/frames.ts` is the `bytes/` route, not moq's). The codec's WASM route stays in the benchmark as the baseline of compiled Rust in a browser. The codec's differential harness is the pattern to repeat. The shared npm runtime @@ -55,6 +62,12 @@ refusals) translation refuses for good, so moq rewrites or drops them, as it doe `SmallVec` an `rs.Vec` (decided with the maintainer on 2026-11-07, in the quest audit, under rule 2 of [m3's unsafe policy](/quest/m3/README.md); rejected: moq dropping smallvec). +Not all of the rest is moq's. moq-net tests identity with +`std::ptr::eq(owner.as_ptr(), self)` on a `Weak` (`model/group/content.rs` at the pin), +which is safe Rust, and translation maps `Arc::as_ptr` but refuses `Weak::as_ptr`. Map +it here, as pointer identity already translates as JS identity; that is on the gate's +path. + Decided with the maintainer on 2026-11-04, from a census at the bumped pin (moq-dev/rust2#390): the refusals no quest owned became child quests, since finished. ✅ arrayvec's `ArrayString` needs three of them (const generic types, `begin_panic`, diff --git a/quest/m2/ubrn-release.md b/quest/m2/ubrn-release.md deleted file mode 100644 index 78a0af9cf..000000000 --- a/quest/m2/ubrn-release.md +++ /dev/null @@ -1,19 +0,0 @@ -# [XS] Waiting: ubrn releases UniFFI 0.32 support - -## Goal - -uniffi-bindgen-react-native (ubrn) publishes a release whose `wasm2` flavour -reads UniFFI 0.32 metadata, so the [binding benchmark](/bench/ffi/README.md)'s -browser runner pins a release instead of an open PR's head. - -## Plan - -Then bump `[ubrn]` in `bench/ffi/pins.toml` to the release in its own change, -and take `@ubjs/core`, `@ubjs/wasm`, and `uniffi-runtime-wasm` from npm and -crates.io instead of building them from source. If ubrn stalls, carrying its TypeScript generator in the fork is the -alternative; that is the maintainer's call. - -## Waiting - -- jhugman/uniffi-bindgen-react-native: [#468](https://github.com/jhugman/uniffi-bindgen-react-native/pull/468) - or a successor merges, and a tag after `0.31.0-6` carries it (neither on 2026-11-07). diff --git a/quest/m2/uniffi-swift-arm.md b/quest/m2/uniffi-swift-arm.md index 29acd1fe1..fdcfaedf8 100644 --- a/quest/m2/uniffi-swift-arm.md +++ b/quest/m2/uniffi-swift-arm.md @@ -24,5 +24,7 @@ Public API: none. - NixOS/nixpkgs: the latest `nixpkgs-unstable` carries Swift 6.4.0 ([NixOS/nixpkgs#574002](https://github.com/NixOS/nixpkgs/pull/574002), merged to - `staging` on 2026-10-22, not in `nixpkgs-unstable` on 2026-11-07). Check that + `staging` on 2026-10-22 and to `master` on 2026-11-07 through + [NixOS/nixpkgs#580394](https://github.com/NixOS/nixpkgs/pull/580394), not in + `nixpkgs-unstable` on 2026-11-09). Check that `nix build --dry-run .#devShells.aarch64-linux.uniffi` succeeds after a bump. diff --git a/quest/m2/wasm/README.md b/quest/m2/wasm/README.md new file mode 100644 index 000000000..88d5d86f2 --- /dev/null +++ b/quest/m2/wasm/README.md @@ -0,0 +1,57 @@ +# rust2-wasm: a crate's pub API in the browser + +## Goal + +[rust2-ffi](/ffi/README.md) exports a crate's `pub` API to WebAssembly with a generated +TypeScript binding, by the rules its other languages follow: no attributes, no config, no +wasm-bindgen, and no step after `cargo build`. The same package loads in browsers, Node, +Bun, and Deno. It is moq's browser route, and the baseline +[rust2-ts](/ts/README.md) has to beat. + +What stays here once every child has merged, the gate: moq-net runs in a browser through +rust2-wasm, and one report compares the same `pub` API through rust2-wasm, through +rust2-ts's translation, and hand-written (`@moq/net`), on CPU time, shipped bytes, and +startup, in Chrome and Firefox. The maintainer decides from it whether translation +continues, and for which crates. + +## Plan + +Decided with the maintainer on 2026-11-09, in the quest audit: + +- ✅ WASM ships, and TypeScript must beat it: rust2-ts is "an experiment to see if it can be + an improvement over WASM", and wasm-bindgen "is kind of annoying to work with". This + supersedes moq's 2026-10-12 "WASM is transitional and JS-only is the final goal" + ([moq's quest](https://github.com/moq-dev/moq/blob/main/quest/m5/rs2ts-js.md)) and + [the model epic](/quest/m2/ts-model/README.md)'s 2026-11-01 "the translated model is + moq's browser route". Rejected: JS-only staying the goal, with rust2-wasm only a + baseline; pausing rust2-ts until rust2-wasm is measured. +- ✅ m2, with [per-target export](/quest/m1/ffi-export/target.md) in m1. Rejected: all of + it in m1; all of it in m2. +- ✅ Until the gate reports, translator work is only what the gate needs: the model's + remaining refusals and the benchmark. [m3](/quest/m3/README.md#plan) records what waits. + Rejected: continuing every translator quest in parallel; freezing the model's work too. + +The gate sets no threshold, as the other benchmarks set none: rust2-go's "within 2x of +cgo" was the maintainer's call from its report too. It compares every workload all routes +run, the codec's today and the model's once it translates, so it need not wait on moq's +[model rewrite](https://github.com/moq-dev/moq/blob/main/quest/m5/rs2ts-model-subset.md). + +Why this shape, from the audit's survey of 2026's tools: + +- wasm-bindgen is maintained (0.2.130 on 2026-10-30), so the case against it is its + design: an attribute on every export, a CLI locked to the crate's version, and glue per + value. Diplomat already runs a C ABI over `wasm32-unknown-unknown` with generated JS. +- Not the Component Model: it is phase 1 with no browser engine behind it, runs in + browsers only through `jco transpile`, and its sync calls cost about 3.5 times a plain + one today. The exporter's IR could emit WIT later; nothing plans it. +- This is also how [m3's unsafe policy](/quest/m3/README.md) compiles an `unsafe`-heavy + crate behind its public API (rule 4). + +moq's [`rs2ts-js`](https://github.com/moq-dev/moq/blob/main/quest/m5/rs2ts-js.md) and +[`rs2ts-wasm2js`](https://github.com/moq-dev/moq/blob/main/quest/m5/rs2ts-wasm2js.md) still +record the superseded decision; they are moq's tree's to update. + +## Required + +- [The exporter's wasm32 target](/quest/m2/wasm/backend.md) - the glue crate builds for wasm32 with a generated TypeScript binding, proven on the api fixture +- [Every route on the browser benchmark](/quest/m2/wasm/routes.md) - `just ts bench` runs rust2-wasm's export and a `wasm2js` control beside the translation diff --git a/quest/m2/wasm/backend.md b/quest/m2/wasm/backend.md new file mode 100644 index 000000000..670ba6042 --- /dev/null +++ b/quest/m2/wasm/backend.md @@ -0,0 +1,67 @@ +# [L] The exporter's wasm32 target + +## Goal + +`rust2-ffi --target wasm32-unknown-unknown` writes, beside the glue crate, a TypeScript +binding over the `.wasm` that `cargo build` makes of it: ES modules a bundler takes as +is, typed as the other languages' bindings are, with no wasm-bindgen and no tool run on +the `.wasm` afterwards. The `api` fixture's script runs through it in Node, Chrome, and +Firefox, its transcript matching native Rust's line for line, with its futures checked as +the other languages' are. The fixture spawns a thread (`ffi/fixtures/api/src/lib.rs`), +which wasm32 cannot, so what needs one moves to a fixture of its own that the native +languages keep running. + +## Plan + +Decided with the maintainer on 2026-11-09: planned in +[the rust2-wasm epic](/quest/m2/wasm/README.md). The rules every generator follows +([the exporter epic](/quest/m1/ffi-export/README.md)) apply: the contract check at load, +bulk payloads, native futures, lossless types, and checks that raise. + +Recommended design, from the audit's survey, for the implementer to confirm by +measurement on [the benchmark](/quest/m2/wasm/routes.md): + +- Records are read from and written to linear memory at the layouts rustc computed for + the target, with no serialization and no JS object per field until one is read. +- A `Uint8Array` view of linear memory detaches whenever memory grows, which any call + into Rust can cause, one a callback makes while it holds lent bytes included. So bytes + Rust returns or lends are an object of their pointer and length that makes its view + from the current `memory.buffer` at each read, and hands out a copy where a caller + wants a `Uint8Array` to keep. Test a callback that grows memory while it reads what it + was lent. Bytes passed in are copied once into memory the call frees. Text is one `TextDecoder` or `TextEncoder` call; JS string builtins do not help + UTF-8 held in linear memory. +- A handle is an integer in a class with `[Symbol.dispose]`, and a `FinalizationRegistry` + frees one never disposed, as the other languages' garbage collectors do. +- A future is a `Promise`: the binding polls once at the call, and an imported `wake` + schedules the next poll on a microtask, never inside the wake. Cancelling frees the + future and loses nothing. JSPI is not needed, since no Rust code blocks on JS. +- One thread: no atomics, no shared memory, no COOP or COEP headers. A type that is not + `Send` exports with no check ([not-send](/quest/m1/ffi-export/not-send.md)). +- The runtime's `fail` writes its message through an import, since wasm32 has no stderr. +- A struct passed by value is passed indirectly in wasm's C ABI, which the binding must + match from the target's layouts, not the host's. + +Open, for the maintainer once the implementer has the facts: + +- What std's parts that wasm32-unknown-unknown lacks do. The file system, sockets, and + `thread::Builder::spawn` return errors there, which is the target's Rust and needs + nothing. `Instant::now` and `SystemTime::now` panic, and moq-net reaches them: + `Timestamp::now` reads kio's clock, whose fallback is `Instant::now`, so refusing the + call would refuse the gate's own consumer. Options: build for `wasm32-wasip1` instead, + whose std has a clock, randomness, and stderr through a handful of imports the + generated binding implements (`performance.now`, `crypto.getRandomValues`), a platform + boundary AGENTS.md allows; refuse at export, with the call's span, any exported item + whose body reaches a call that can only panic, as rust2-ts already reads every body's + MIR; or moq gives kio a wasm32 clock, a source change AGENTS.md calls a gap. + Recommended: the first, if its imports stay that few and its `.wasm` no larger; + measure both targets on the fixture before choosing. +- How a crate that itself imports browser APIs through wasm-bindgen (web-sys) exports. + Recommended: refuse it at the import's span until a consumer needs both. + +Public API: the TypeScript binding's shape, and `--target wasm32-unknown-unknown`. + +## Required + +- [Export for a target](/quest/m1/ffi-export/target.md) - the exporter checks and lays out the API for wasm32, not the host +- [Lossless binding types](/quest/m1/ffi-export/lossless.md) - the types a new binding copies are the exact ones +- [Failed checks raise](/quest/m1/ffi-export/checks-raise.md) - the one accounting a new binding copies raises diff --git a/quest/m2/wasm/routes.md b/quest/m2/wasm/routes.md new file mode 100644 index 000000000..5d93b075f --- /dev/null +++ b/quest/m2/wasm/routes.md @@ -0,0 +1,38 @@ +# [M] Every route on the browser benchmark + +## Goal + +`just ts bench` ([the TS benchmark](/ts/README.md#bench)) times each workload through two +more routes, in Chrome, Firefox, Node, and Bun: the fixture's `pub` API exported by +[rust2-wasm](/quest/m2/wasm/backend.md), called through its generated binding, and the +same `.wasm` lowered to JavaScript by Binaryen's `wasm2js`. Each route's output must match +native Rust's first, and the report gives each route's shipped bytes, raw and gzipped, +with its `.wasm` or runtime included. + +## Plan + +Decided with the maintainer on 2026-11-09: this is the measurement +[the gate](/quest/m2/wasm/README.md) reads. + +- The benchmark's existing wasm32 route stays, as compiled Rust with a hand-written + boundary whose copies are timed apart: against it, the rust2-wasm route shows what the + generated binding costs. +- `wasm2js` is the control no one has measured: it needs no translator and no WebAssembly + at run time, which is what moq's + [`rs2ts-wasm2js`](https://github.com/moq-dev/moq/blob/main/quest/m5/rs2ts-wasm2js.md) + planned to try first. No published speed or size figures for it were found. If it + matches the translation on speed and size, that is a finding about the translator, to + report plainly. +- What is published points both ways, so only this benchmark decides: a Rust and WASM + parser lost 2.2 to 4.6 times to a TypeScript rewrite on boundary costs + ([OpenUI, 2026-03](https://www.openui.com/blog/rust-wasm-parser)), while Scala.js on + Wasm won up to 5.8 times on hashing and lost on object-heavy workloads + ([VirtusLab, 2024-07](https://virtuslab.com/blog/scala/scala-to-web-assembly/)). +- Add moq's `js/net/bench/frames.ts` workload, a group stream read into a group, once the + model translates; `ts/bench/frames.ts` is the `bytes/` route, not moq's. + +Public API: none. + +## Required + +- [The exporter's wasm32 target](/quest/m2/wasm/backend.md) - the binding the new route calls through diff --git a/quest/m3/README.md b/quest/m3/README.md index 72adcb369..1951d78bf 100644 --- a/quest/m3/README.md +++ b/quest/m3/README.md @@ -2,14 +2,27 @@ ## Goal -Work that waits on evidence or has no consumer yet: the build-speed tracks, -each a measurement that decides whether its follow-up lands, and the parts of -std the translators cover before a consumer calls them. +Work that waits on evidence or has no consumer yet: two waits on upstream's build-speed +levers, the translator's features that wait on [the gate](/quest/m2/wasm/README.md), the +binding targets planned ahead of a consumer, and what waits on moq. ## Plan -Decided with the maintainer on 2026-10-27: rust2 aims to support all of std, so a -generic std gap is planned here even when moq never calls it. +Decided with the maintainer on 2026-11-09, in the quest audit: + +- ✅ Until [the gate](/quest/m2/wasm/README.md) reports, translator work is only what the + gate needs. The quests here that no consumer calls wait on it: + [no `ts.toml`](/quest/m3/ts-configless.md) and + [browser APIs through wasm-bindgen](/quest/m3/ts-wasm-bindgen/README.md). This + supersedes, until then, the 2026-10-27 decision that rust2 aims to support all of std, + so a generic std gap was planned here even when moq never calls it. Rejected: + continuing as before; freezing the model's work too. +- ✅ C# and Unity, a Node-API addon, and React Native are planned here before any consumer + asks, since they are the targets a media SDK's users expect: LiveKit ships Python, Node, + Unity, and React Native over its Rust core. Each waits on + [the exporter epic](/quest/m1/ffi-export/README.md), so it copies the settled C ABI, + types, and checks. Rejected: none until a consumer asks. Not planned: Java without + Kotlin (the JNI library serves it, and Android has no FFM API), Ruby, PHP, Elixir, Lua, and game engines. Decided with the maintainer on 2026-11-07, the unsafe policy, to stop one-off fixes for ever rarer `unsafe`: @@ -23,7 +36,15 @@ ever rarer `unsafe`: or pointer metadata is refused with its span, unless a consumer needs it and no library wrapper covers it. 4. ✅ An `unsafe`-heavy crate a consumer needs is compiled to WASM behind its public API - rather than fixed construct by construct. Not planned until a consumer needs it. + rather than fixed construct by construct, through + [rust2-wasm](/quest/m2/wasm/README.md) once it exists. Mixing a translated crate with + one compiled so is not planned until a consumer needs it. +5. ✅ Where translation can't see where an `unsafe` pointer came from, a loud run-time + check stands in for the refusal, the exception AGENTS.md names: `ptrUnerase`, + `offset_from` across arrays, `AtIndex.reach`, and raw slices of a field's byte + (decided 2026-11-07 in [#661](https://github.com/moq-dev/rust2/pull/661) and + [#621](https://github.com/moq-dev/rust2/pull/621)). Safe code gets no such check: it + translates exactly or is refused. Rejected: an unsafe std gap with no consumer becoming its own quest, as `MaybeUninit::zeroed`, `Rc::new_uninit`, and `ptr::write_bytes` would have @@ -33,6 +54,10 @@ memory model for values whose address is reinterpreted, without a consumer. ## Required +- [Sessions on sans-IO QUIC](/quest/m3/session-quic.md) - bindings move UDP datagrams and drive the session's poll themselves, over moq's runtime-free QUIC driver once it lands +- [C# and Unity bindings](/quest/m3/csharp.md) - a C# generator over the C ABI that runs under .NET and Unity's IL2CPP +- [A Node-API addon](/quest/m3/node.md) - a native Node binding, kept only if it beats rust2-wasm and `node:ffi` +- [React Native bindings](/quest/m3/react-native.md) - a JSI module over the generated C++ header - [Retrial Cranelift on moq's tests](/quest/m3/cranelift-unwind.md) - waits on upstream: moq's tests pass under Cranelift, so its CI can weigh it again - [Relink without rebuilding](/quest/m3/relink.md) - measure upstream's skip of dependency-only rebuilds on moq's edits once it reaches nightly - [A Pin of a crate's own Deref type](/quest/m3/ts-pin-own-deref.md) - `Pin

` over a crate's own `Deref` derefs through its impl and type-checks diff --git a/quest/m3/csharp.md b/quest/m3/csharp.md new file mode 100644 index 000000000..41bfd8bd7 --- /dev/null +++ b/quest/m3/csharp.md @@ -0,0 +1,36 @@ +# [L] C# and Unity bindings + +## Goal + +[rust2-ffi](/ffi/README.md) writes `csharp/.cs`, the crate's API in C# over the C +ABI, which runs under .NET (NativeAOT included) and under Unity's IL2CPP on iOS and +Android. It follows every generator's rules +([the exporter epic](/quest/m1/ffi-export/README.md)), the `api` fixture's script runs +through it with a transcript matching native Rust's, and it is on +[the binding benchmark](/bench/ffi/README.md). + +## Plan + +Decided with the maintainer on 2026-11-09, in the quest audit: ✅ planned in m3 before a +consumer asks ([m3](/quest/m3/README.md#plan)); moq ships no C# today. + +- Prior art, all annotation-based: csbindgen (reads `extern "C" fn`), Interoptopus, + BoltFFI, Diplomat, and uniffi-bindgen-cs. None exports a `pub` API as it is. +- Recommended: blittable signatures only, called through function pointers + (`delegate* unmanaged[Cdecl]`) so no marshalling stub runs, with records as + `[StructLayout(LayoutKind.Sequential)]` structs at the C layouts, bytes as + `ReadOnlySpan` in and a lent `ReadOnlyMemory` owner out, futures as + `ValueTask` on the caller's `SynchronizationContext`, and handles as `SafeHandle`s with + `IDisposable`. Confirm each against the C# language level and IL2CPP limits of Unity's + current LTS, where callbacks into managed code need `[MonoPInvokeCallback]` and iOS + links the library statically (`__Internal`); where the two runtimes need different + code, the generator writes one file both compile. +- Unity cannot run in this repository's CI. Open: how Unity is proven. Recommended: the + .NET test in the gate, compiled also at Unity's language level, and a nightly IL2CPP + build without the editor if one can be made headless; report what is possible. + +Public API: the generated C# API. + +## Required + +- [Exporter](/quest/m1/ffi-export/README.md) - the C ABI, its lossless types, and its checks are settled before a new generator copies them diff --git a/quest/m3/node.md b/quest/m3/node.md new file mode 100644 index 000000000..772cd0101 --- /dev/null +++ b/quest/m3/node.md @@ -0,0 +1,35 @@ +# [L] A Node-API addon + +## Goal + +A report, then a binding if the report earns it: the `api` fixture and moq's frame +workloads called from Node three ways, through [rust2-wasm](/quest/m2/wasm/README.md), +through Node's built-in `node:ffi` over the existing C header, and through a native +Node-API addon the crate builds with `--features node`, as Python's CPython extension is +built. The addon lands as a generator only where it beats both others on +[the binding benchmark](/bench/ffi/README.md); otherwise this quest is deleted with its +numbers in the PR. + +## Plan + +Decided with the maintainer on 2026-11-09, in the quest audit: ✅ planned in m3 before a +consumer asks ([m3](/quest/m3/README.md#plan)). + +- rust2-wasm already loads in Node, Bun, and Deno, so a native addon must earn its cost + (a 3,000 to 5,000 line generator, and a binary per platform) in speed or in what wasm + cannot do: threads, sockets, and the file system, which a library such as moq-tokio + needs for native QUIC. +- `node:ffi` is experimental since Node 26.1 (2026-05), and Bun and Deno have their own + FFI; Bun's docs call its `bun:ffi` not production-ready. If a runtime's own FFI over the + header is fast enough, a thin generated TypeScript file over it replaces the addon. +- No tool was found that generates Node-API bindings from a C header; napi-rs is + annotation-based. +- Node-API is ABI-stable across Node versions and is what Bun and Deno load too, so one + addon serves all three. + +Public API: the generated TypeScript API and the `node` feature, if it lands. + +## Required + +- [Exporter](/quest/m1/ffi-export/README.md) - the C ABI, its lossless types, and its checks are settled before a new generator copies them +- [The exporter's wasm32 target](/quest/m2/wasm/backend.md) - the route the addon is measured against, whose TypeScript API it should match diff --git a/quest/m3/react-native.md b/quest/m3/react-native.md new file mode 100644 index 000000000..44d5d383e --- /dev/null +++ b/quest/m3/react-native.md @@ -0,0 +1,34 @@ +# [L] React Native bindings + +## Goal + +[rust2-ffi](/ffi/README.md) writes a React Native module for the crate: TypeScript over a +JSI module in C++ that calls the generated C++ header, so a call from Hermes reaches Rust +with no bridge serialization, on iOS and Android. The `api` fixture's script runs through +it on an Android emulator and an iOS simulator, nightly. + +## Plan + +Decided with the maintainer on 2026-11-09, in the quest audit: ✅ planned in m3 before a +consumer asks ([m3](/quest/m3/README.md#plan)). + +- Prior art: uniffi-bindgen-react-native (ubrn) generates JSI C++ and TypeScript from + UniFFI's annotations; its releases trail UniFFI's (no release read UniFFI 0.32 by + 2026-11-09). +- It builds on what exists: `rt.hpp`'s futures and executors for the JS thread's + scheduling, and the TypeScript API [rust2-wasm](/quest/m2/wasm/backend.md) generates, + which this should match so one application's code calls either. +- The JS runtime is single-threaded while Rust wakes from any thread, so a wake posts to + the runtime's `CallInvoker`, never polling inside the wake, as every other binding's does. +- Open: whether rust2-wasm can serve React Native instead. Check first whether Hermes + runs WebAssembly, which the audit did not verify; ubrn generating JSI C++ suggests it + does not. If it does, measure it as [the Node quest](/quest/m3/node.md) does for Node, + and write the JSI module only if it wins or wasm cannot run the library (moq's native + QUIC needs sockets). + +Public API: the generated module's TypeScript API. + +## Required + +- [Exporter](/quest/m1/ffi-export/README.md) - the C ABI, its lossless types, and its checks are settled before a new generator copies them +- [The exporter's wasm32 target](/quest/m2/wasm/backend.md) - the TypeScript API this matches, and the route it is measured against diff --git a/quest/m3/relink.md b/quest/m3/relink.md index ff5ba9409..b88c0bef7 100644 --- a/quest/m3/relink.md +++ b/quest/m3/relink.md @@ -8,7 +8,10 @@ variant on moq's edits. ## Plan -It is the one upstream lever the moq-net edit leaves room for. The benchmark's +It is the one upstream lever the moq-net edit leaves room for. The figures below are from +moq before its poll-first rewrite: [#646](https://github.com/moq-dev/rust2/pull/646) has +since moved the benchmark's pin past it, and numbers across a pin bump do not compare, so +measure again before relying on them. The benchmark's attribution ([run](https://github.com/moq-dev/rust2/actions/runs/40093020939)) puts 4.5 s of that edit's 6.95 s critical path in crates rebuilt only because moq-net changed, 0.6 s of it the final link that would remain. Cranelift and diff --git a/quest/m1/ffi-export/session-quic.md b/quest/m3/session-quic.md similarity index 82% rename from quest/m1/ffi-export/session-quic.md rename to quest/m3/session-quic.md index e5a7cd814..216381455 100644 --- a/quest/m1/ffi-export/session-quic.md +++ b/quest/m3/session-quic.md @@ -5,12 +5,16 @@ A binding runs moq sessions over moq-quic's sans-IO QUIC with no runtime in the library: the library owns QUIC and TLS, and the binding moves UDP datagrams and drives the session's poll from its own event loop, as driving decided for the origin's -`Driver`. Proven from C in `moq/moq.c` against a local server. +`Driver`. Proven from C in `ffi/moq/moq.c` against a local server. ## Plan +Decided with the maintainer on 2026-11-09, in the quest audit: ✅ m3, out of +[the exporter epic](/quest/m1/ffi-export/README.md), which it would have held open until +moq's m3 driver lands. Rejected: m2; a top-level m1 quest. + Decided with the maintainer on 2026-10-27: the full-control path, after sessions on -moq-tokio, which `moq/moq.c` proves with moq-tokio's fallback runtime. It suits consumers +moq-tokio, which `ffi/moq/moq.c` proves with moq-tokio's fallback runtime. It suits consumers with their own event loop or sockets. Decided with the maintainer on 2026-10-28, from diff --git a/quest/m3/ts-configless.md b/quest/m3/ts-configless.md index c30ce45fe..562bb5297 100644 --- a/quest/m3/ts-configless.md +++ b/quest/m3/ts-configless.md @@ -8,6 +8,9 @@ package is the crate it runs on, every `pub` item is a root, a `pub` type's `Dis ## Plan +Decided with the maintainer on 2026-11-09, in the quest audit: this waits on +[the gate](/quest/m2/wasm/README.md), which decides whether translation continues. + Decided with the maintainer on 2026-11-04: every tool is configless, with opinionated names and defaults. @@ -22,8 +25,20 @@ names and defaults. Fixtures that name a few roots become crates whose `pub` API is those roots. Split into an epic if the binds alone outgrow one PR. +The model translating is not moq-net translating whole: at `f7a0c279` the nightly census +refused 328 items of all of moq-net and 71 of kio (form_urlencoded, tokio's runtime, +zlib-rs, arrayvec), against 41 from the model's roots. No quest owns that gap; plan it +when this starts. + +Recommended: `--cfg rust2_ts` goes too. `ts/src/profile.toml` still sets it so a crate +can give translation a safe twin of its `unsafe` (`ts/README.md`), which is a source +change AGENTS.md calls a gap, and moq writes no twins +([the model epic](/quest/m2/ts-model/README.md)); only `ts/fixtures/crates/dep` and +`ts/fixtures/encoder` use it. + Public API: the command line, and what a translation exports. ## Required - [moq-net's model in TypeScript](/quest/m2/ts-model/README.md) - the model translates with no refusals +- [rust2-wasm](/quest/m2/wasm/README.md) - its gate decides whether translation continues past the model diff --git a/quest/m3/ts-wasm-bindgen/README.md b/quest/m3/ts-wasm-bindgen/README.md index 14392aa8f..f0da460ab 100644 --- a/quest/m3/ts-wasm-bindgen/README.md +++ b/quest/m3/ts-wasm-bindgen/README.md @@ -6,8 +6,11 @@ (web-sys, js-sys, wasm-bindgen-futures, and crates built on them such as [web-rs](https://github.com/kixelated/web-rs)) from source with no `ts.toml` binding, each import becoming a direct JS call, so Rust developers keep using idiomatic Rust crates -instead of JS types. web-rs's web-codecs translates end to end, a `VideoEncoder` included, -which is this epic's own remaining work. +instead of JS types. web-rs's web-codecs translates end to end as a stock crate, its +decoders and audio included, which is this quest's remaining work: the +[encoder fixture](/ts/fixtures/encoder/Cargo.toml) proves a `VideoEncoder`'s calls on a +copy of web-codecs' encoder ([#681](https://github.com/moq-dev/rust2/pull/681)), not on +the crate. Non-goals: `#[wasm_bindgen]` exports, since a translation already exports the crate's `pub` API, and imports of custom JS (`module`, `raw_module`, `inline_js`), which no consumer uses @@ -15,6 +18,9 @@ yet. Both are refused at translation with their spans. ## Plan +Decided with the maintainer on 2026-11-09, in the quest audit: the remainder waits on +[the gate](/quest/m2/wasm/README.md), since no consumer calls it. + Decided with the maintainer on 2026-11-06: - ✅ Translate wasm-bindgen's imports (rejected: mapping each browser API into the runtime @@ -29,7 +35,9 @@ Decided with the maintainer on 2026-11-06: `method`, `js_class`, `js_name`, `js_namespace`, `getter`, `setter`, `constructor`, `static_method_of`, `catch`, `extends`, `variadic`, `indexing_getter`, `indexing_setter`, `indexing_deleter`, `typescript_type`, `is_type_of`, `final`, `structural`, - `thread_local_v2`, and `vendor_prefix`. js-sys's JSPI bridge also declares a + `thread_local_v2`, and `vendor_prefix`. Three things those crates use are refused, + each at its span (`ts/README.md`): an `experimental_generic_mono` import, a mutable + slice within an `Option`, and 0.2.130's `Nullable`. js-sys's JSPI bridge also declares a `suspending` import, which suspends the WASM stack on a promise. No JS call can, so it is refused where reached, as is `jspi_block_on_promise`, which suspends on it. No translated path reaches it: `spawn_local` is mapped, and no JSPI context exists. @@ -56,3 +64,7 @@ Decided with the maintainer on 2026-11-06: [tokio's sync channels are mapped into the runtime](/ts/src/lower/tokio.rs) (rejected: translating tokio's source, whose intrusive lists and raw pointers into `UnsafeCell`s are unsafe code a JS caller has no use for). + +## Required + +- [rust2-wasm](/quest/m2/wasm/README.md) - its gate decides whether translation continues past the model diff --git a/quest/m4/fork.md b/quest/m4/fork.md index 5b5822bc5..16cc1c470 100644 --- a/quest/m4/fork.md +++ b/quest/m4/fork.md @@ -11,7 +11,7 @@ Later reports append their data points here. - The fork carries its changes itself rather than working through upstream. - Measure each reason on moq's edits in the [build-speed benchmark](/bench/build/README.md). -- Code that follows the subset's rules (poll-first, thin generics) only gains +- Code that follows moq's poll-first and thin-generics rules only gains more; it is never required. ### Reasons to fork @@ -50,6 +50,6 @@ Public API: none until the fork has a consumer. ## Waiting -- Maintainer: decide that the reasons to fork below add up to enough to pay for +- Maintainer: decide that the reasons to fork above add up to enough to pay for maintaining one. An agent can only gather the evidence; the cost of carrying a fork is the maintainer's to weigh. diff --git a/quest/m4/go/coverage.md b/quest/m4/go/coverage.md index 7457a0e76..0d4e5eaae 100644 --- a/quest/m4/go/coverage.md +++ b/quest/m4/go/coverage.md @@ -21,7 +21,9 @@ The spike refuses each of these with its span (`go/fixtures/reject/expected.txt` or types, which needs one Go variable and references derived into it by its layout. - **`Vec::capacity`,** which Go's growth would make differ from Rust's. - **std's collections:** `HashMap`, `BTreeMap`, `VecDeque`, translated from std's MIR or - bound to the runtime as rust2-ts's are. + bound to the runtime as rust2-ts's are. The reject fixture holds no case for them yet. +- **Statics and pointers the fixture also refuses:** a `static mut`, a static with + interior mutability such as an atomic, and `NonNull` ([go/README.md](/go/README.md)). The translation targets the host, 64-bit `usize`; a 32-bit Go target needs its own translation.