Repository navigation
feat(bb-avm-sim): a wasm backend for the bb-avm-sim package - #25670
charlielye wants to merge 7 commits into
Conversation
Moves `noir/noir-repo` from `v1.0.0-rc.3` (`5a7ee9bf`) to `v1.0.0-rc.5` (`0f759ffe`). Produced by `noir/scripts/bump_noir_compiler.sh v1.0.0-rc.5`: the submodule pointer and `avm-transpiler/Cargo.lock` (noir path packages only). `noir-projects/fnd` formatting with the rc.5 `nargo` produced no changes. Upstream range: noir-lang/noir@v1.0.0-rc.3...v1.0.0-rc.5 (162 commits, none marked breaking) ### Labs patch `0019` rc.5's `nargo expand` prints associated-constant array lengths in contract interface stubs as written rather than folded (`[Field; (16 * 20) + 1]` and `Serialize<N = (16 * 20) + 1>` instead of `[Field; 321]`). Five `contract-snapshots` expand snapshots (`amm`, `token`, `avm_gadgets_test`, `public_fns_with_emit_repro`, `storage_proof_test`) are regenerated for that. It is a test-only change, carried as a labs patch. Notable upstream changes that could move artifacts here: - Many `fix(frontend)` trait-constraint/associated-type resolution fixes (e.g. [noir-lang/noir#13982](noir-lang/noir#13982), [#13971](noir-lang/noir#13971), [#13939](noir-lang/noir#13939)). - `fix(ssa)`: Brillig missing-constraints check refinements ([#13893](noir-lang/noir#13893), [#13886](noir-lang/noir#13886), [#13879](noir-lang/noir#13879), [#13880](noir-lang/noir#13880)). - `fix(acir)`: constant-length `vector_push_back` growth ([#13945](noir-lang/noir#13945)) and predicate-zero blackbox calls in ACIR gen ([#13965](noir-lang/noir#13965)). Checked locally: `nargo` and `avm-transpiler` build, the fnd formatter makes no changes, `contract-snapshots` passes (61/61) against this tree with rc.5, and `labs-patches/bootstrap.sh check` applies the series cleanly. --- *Created by [claudebox](https://claudebox.work/v2/sessions/be9a837cc5ef642a/jobs/1) · group: `slackbot` · requested by Tom (@TomAFrench) · [Slack thread](https://aztecfoundation.slack.com/archives/D0B586H14KG/p1792071360555509?thread_ts=1792071360.555509&cid=D0B586H14KG)*
Add `v6` to the nightly release tag matrix so it gets `v6.0.0-nightly.<date>` tags alongside `next` and `v5-next`. The release line is `v6` (there is no `v6-next`), and its manifest version does not collide with the other branches' tags.
Every bb wasm build compiled with -fno-exceptions, so throw_or_abort called out to the JavaScript environment, whose throw unwound the module without running catch blocks or destructors, and the AVM simulator (which signals exceptional halts with C++ exceptions) could not build for wasm at all. wasi-sdk 33+ ships a sysroot with a C++ exception runtime, so the wasm and wasm-threads builds now use standard (exnref) wasm exception handling, and the toolchains move to versions that run it unflagged. bb wasm - arch.cmake: -fwasm-exceptions -mllvm -wasm-use-legacy-eh=false, linked with -lunwind, in place of -fno-exceptions. The wasm preset drops -DBB_NO_EXCEPTIONS and sets AVM off. - throw_or_abort throws std::runtime_error inside the module in wasm builds with exceptions, so bbapi answers with an ErrorResponse, as it does natively. bb.js's exception test now also checks the module no longer aborts (it fails on the old wasm). - The wasm build adds aztec, world_state_reference, vm2 (vm2_sim only) and crypto/merkle_tree, and the wasm build preset builds vm2_sim_smoke. That runs one hinted public transaction (vm2/testing/minimal_tx.testdata.bin) through AvmSimAPI::simulate_with_hinted_dbs after checking a TagMismatchException is caught; scripts/vm2_sim_smoke_test.sh runs it natively and in wasm under node and requires identical results. It runs in the default bb test_cmds. - wasm32 fixes in the simulator and the in-memory trees, where size_t is 32-bit: contract_crypto shifted a size_t by 32, which is undefined on wasm32 and would corrupt the first bytecode field; index_t to size_t casts in the in-memory indexed trees and their callers; ptrdiff_t in to_radix. - LLVM 23 (wasi-sdk 34) fixes: wasm32-wasip1-threads in place of the removed wasm32-wasi-threads target; missing <algorithm> and <cstdlib> includes; static function templates in headers made inline (-Wunused-template); -Wno-c2y-extensions for google-benchmark, which builds with -pedantic-errors and uses __COUNTER__. - barretenberg.wasm grows 5% (17.17 MB to 18.03 MB), 3.4% gzipped (3.88 MB to 4.01 MB). UltraHonk 2^20 proving under wasmtime showed no difference beyond noise. Toolchains (build image and scripts/setup-container.sh) - node 24.15.0 to 26.12.0 (.nvmrc, bootstrap's expected minimum and install_node). Node 26 removed --experimental-transform-types; the ipc-codegen callers only need type stripping, so it is dropped. - wasi-sdk 27 to 34 at /opt/wasi-sdk. - wasmtime 20.0.2 to 46.0.3, the last release with wasi-threads (-Sthreads), which the threaded wasm tests and benchmarks use. Its invocations add -Wexceptions=y and -Wshared-memory=y. - playwright 1.49 to 1.63.0 (Chromium 153; exnref is unflagged from Chrome 137), in the image and headless-test. The CI image is not redeployed here. Until it is, the root bootstrap's transitional_toolchain_upgrade (run by prep and check, in CI or the devbox) installs wasi-sdk 34 at /opt/wasi-sdk and node, wasmtime and playwright's chromium under $HOME, passing PATH and PLAYWRIGHT_BROWSERS_PATH through to isolated test containers. In devbox:3.1 it takes about 36s the first time and 3s after. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01G5s7nEWrYapDK2CWpEzcAf
This PR keeps to the toolchain and exception handling. The vm2_sim_smoke program, its test, the vm2-in-wasm CMake and the wasm32 fixes in the simulator move to the PR that builds bb-avm-sim as a wasm module for its generated package. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01G5s7nEWrYapDK2CWpEzcAf
On this base bb's wasm env exits the module on throw_or_abort, and bb.js pinned a thrown error as a plain Error on wasm. With exceptions it is a BbError on both backends, and the module keeps answering. Drops the node 26-removed --experimental-transform-types from the bb.js-api codegen. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01G5s7nEWrYapDK2CWpEzcAf
bb-avm-sim.wasm is the AVM simulator as a WASI reactor exporting the generated FFI entry (avm_ipc_ffi_entry), so @aztec-foundation/bb-avm-sim runs the spawned binary in node and the wasm module in browsers, as bb.js-api does for bb. The module has no route to world state or contract data yet, so it serves SimulateWithHints and answers Simulate with an error. bb-avm-sim's --wsdb and --cdb become optional on the same terms. - avm_ffi: the FFI entry and its context, built natively and for wasm. avm_ffi_tests calls the entry with vm2's minimal_tx fixture and checks the result against the public inputs it carries; it runs natively and, from the threads build, under wasmtime like ecc_tests. - The wasm build adds aztec, world_state_reference, vm2 (the simulator alone, AVM off) and avm, with the simulator's wasm32 fixes (a 32-bit shift in contract_crypto, size_t narrowing in the indexed trees). - The package's test runs one transaction through the binary, the wasm worker and the synchronous wasm service and requires the same result. - ipc-runtime's prepare_arch_packages reads a string bin, the form yarn rewrites the generated one-entry bin map to on install; it staged the binary as a file named 0. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01G5s7nEWrYapDK2CWpEzcAf
bb links its wasm with --import-memory. Under wasi-threads wasmtime makes that memory, but for a single-threaded build the CLI fills no env.memory, so the tests ran from the threads build. wasmtime.sh's SINGLE_THREADED=1 now preloads a module exporting one, and avm_ffi_tests runs from the build bb-avm-sim.wasm comes from. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01G5s7nEWrYapDK2CWpEzcAf
|
Review the following changes in direct dependencies. Learn more about Socket for GitHub.
|
|
Warning Review the following alerts detected in dependencies. According to your organization's Security Policy, it is recommended to resolve "Warn" alerts. Learn more about Socket for GitHub.
|
Stacked on #25433 (the wasm transport in ipc-runtime) and #25669 (wasm exception handling, node 26, wasi-sdk 34), which is merged in here. Review those first; this PR is the last two commits.
Why
@aztec-foundation/bb-avm-simcould only spawn the nativebb-avm-simbinary. With exceptions now on in wasm, the AVM simulator builds for wasm, so the package can run it in-process the way bb.js-api runs bb: the binary in node, the wasm module in browsers. This is the first step towards running public simulation in a browser node.What changes
C++ (
barretenberg/avm)avm_schema.jsonis now also generated with--ffi, giving the in-process entryavm_ipc_ffi_entry.avm_ffimodule: that entry, its dispatcher context and a sharedsimulate_with_hints. It builds natively and for wasm.bb-avm-sim.wasm(1.6 MB): a WASI reactor overavm_ffi, built by thewasmpreset like wsdb-ref's module.SimulateWithHints(whose hints answer every read) and answersSimulatewith an error. Reaching wsdb and cdb from wasm is thehost_callfollow-up.--wsdband--cdbare now optional. Without them it servesSimulateWithHintsalone, on the same terms as the wasm module.AVMoff) and avm. It brings the simulator's wasm32 fixes: a 32-bit shift incontract_cryptoand size_t narrowing in the indexed trees.Tests
avm_ffi_testscalls the FFI entry with vm2'sminimal_txfixture, embedded at configure time, and checks the result against the public inputs the fixture carries (gas, fee, revert status, effect counts). It also checks that simulator exceptions are caught, that a malformed input gets an error and the next call still works, and thatSimulateanswers with an error.*_tests, and under wasmtime from the single-threaded build (test_cmds_wasm,make bb-cpp-wasm-tests).--import-memory. The wasmtime CLI only supplies that memory under wasi-threads, sowasmtime.shgainsSINGLE_THREADED=1, which preloads a one-line module exporting it (scripts/wasm_env_memory.wat).barretenberg/ts/scripts/bb_avm_sim.test.mjs(make bb-avm-sim-tests) runs one transaction through the spawned binary, the wasm worker and the synchronous wasm service, and requires the same result, matching the fixture's public inputs. Errors come back asAvmErrorand the service keeps answering.Package and build
--package-transports uds,wasm --package-wasm-module bb-avm-sim.wasm.bootstrap.shcopies the module in for builds and releases, andbb-avm-simnow depends onbb-cpp-wasm.prepare_arch_packagesnow handles a stringbin.yarn installrewrites the generated one-entrybinmap to that form, and the script then staged the native binary as a file named0.Testing
avm_ffi_tests: 4/4 natively (clang20), 4/4 under wasmtime 46 from the single-threaded build and from the threads build.bb_avm_sim.test.mjson node 26: 4/4 (binary, wasm worker, sync wasm).🤖 Generated with Claude Code
https://claude.ai/code/session_01G5s7nEWrYapDK2CWpEzcAf