Skip to content

feat(builtins): run real uutils coreutils as wasm guests (wasm-coreutils) - #2656

Draft
chaliy wants to merge 14 commits into
mainfrom
claude/project-thread-8q1c3m
Draft

chaliy wants to merge 14 commits into
mainfrom
claude/project-thread-8q1c3m

Conversation

@chaliy

@chaliy chaliy commented Oct 9, 2026 •

Copy link
Copy Markdown
Contributor

Requested by Михайло · project thread

What changed

  • New experimental feature wasm-coreutils (off by default). Bashkit can now run real uutils/coreutils 0.12 programs. The 70 utilities are compiled to wasm32-wasip1, ahead-of-time compiled to Pulley in build.rs, and embedded in the binary.
  • Bash::builder().wasm_coreutils() registers a coreutils <util> [args...] multicall (plus coreutils --list) and fills in the 7 utilities bashkit has no native builtin for: csplit, dir, dircolors, pathchk, ptx, shred, vdir.
  • wasm_coreutils_replace_native(limits) routes every embedded utility to the uutils program. It gives GNU-compatible output (for example sort -h) and is useful for differential testing.
  • The CPython WASI host moved to a shared builtins/wasi_host/ and is used by both CPython and coreutils:
    • one runner covering engine, pool, fuel, deadline and budget polling, and exit/trap mapping;
    • an optional read-only overlay;
    • an HTTP bridge that is compiled only with cpython.
  • The new bytes-only crate bashkit-coreutils-wasm ships the precompiled module. Its reproducible guest build lives in guest/ (build.sh, pinned rustc 1.95.0, MANIFEST with hashes).
  • Benchmarks:
    • benches/wasm_coreutils.rs compares native and wasm and checks that their outputs match before timing.
    • examples/wasm_coreutils_startup.rs plus just bench-wasm-coreutils measure fresh-process startup.
  • Knowledge and docs:
    • knowledge: knowledge/runtimes/wasm-coreutils.md, threat model TM-WCU-001..008, limitations L-WCU-001..004;
    • rustdoc guide docs/wasm-coreutils.md;
    • CI test step and publish job.

Security audit and hardening

The WASI host was audited and every limit was attacked with hostile scripts. No sandbox escape was found. These gaps were found and fixed:

Gap Before After
Work budget charged after the call, a long guest ran to its deadline fuel starts at the remaining work units; the guest stops mid-run
Waits nothing wakes (pool slot, cancel) ran past the deadline re-checked every 50 ms, exit 124
writev with many iovecs over one buffer copied in full into host memory before the output cap only what the cap or file size limit admits is copied
Huge paths, big directories path copied before the length check; listing cloned per readdir refused unread; listing cached and charged to the call's buffer budget
/dev/null buffered like a file discard sink
FS limit errors EIO EFBIG, ENOSPC, ENAMETOOLONG, ELOOP
builtin_filter coreutils rm and guest fill-ins bypassed it applies to guest utilities and the multicall; embedder builtins win
Script analysis coreutils rm -rf / hid rm coreutils is reported as a command wrapper
Clock guests read host time (file mtimes too) virtual clock (fixed_epoch), 100 ms resolution under Hardened
Pool starvation one tenant could take all 512 slots max_concurrent per Bash (default 4), waited for within the deadline
Panics dd panicked (thread spawn); Rust panic text could reach stderr dd dropped from the guest; panic hook prints <util>: internal error
Exit status not masked 8 bits

Accepted and documented: the build-time BASHKIT_COREUTILS_CWASM override (TM-WCU-008), and CPython still reads the host clock (TODO in builtins/cpython/mod.rs).

Why

Users want real, GNU-compatible programs instead of reimplementations. Bashkit also has to keep many tenants isolated in one process and start fast.

Native ELF binaries in-process (microsoft/litebox) were evaluated and rejected, for three reasons:

  • LiteBox installs a process-wide seccomp filter.
  • Guest code would share the host address space.
  • It needs global signal state.

Wasm keeps the memory boundary in-process on every platform and compiles nothing at run time.

Before

$ bashkit -c 'pathchk -p report.txt'
bash: pathchk: command not found
$ bashkit -c "printf '1K\n3M\n2\n' | sort -h"   # native sort, no GNU -h semantics

After

let mut bash = Bash::builder().wasm_coreutils().build();
bash.exec("pathchk -p report_2026.txt && echo ok")                 // ok
bash.exec("printf '1K\\n3M\\n2\\n' | coreutils sort -h")          // 2 / 1K / 3M
bash.exec("cd /d && coreutils sort -r f | coreutils head -n 2")    // VFS cwd + pipelines

Measured on a 4-vCPU x86-64 VM with a release build (crates/bashkit/benches/results/criterion-wasm-coreutils-vm-linux-x86_64-1791511758.md):

Measure Native builtin Wasm (uutils on Pulley)
First call, fresh process (engine + module load) 0.04 ms 6.7 ms
echo/cat/wc/sha256sum, small, warm 10-25 µs 2.3-3.9 ms
sort, 100 lines 32 µs 7.8 ms
cat | sort -r | head 105 µs 14.9 ms
cat 1 MiB 519 MiB/s 165 MiB/s
wc / sha256sum 1 MiB ~187 MiB/s 4.5-5.2 MiB/s
sort 1 MiB 56 MiB/s 1.4 MiB/s
64 concurrent sessions 30.5 k/s 322 /s

Instantiation takes about 9 µs.

About 97% of the per-call floor is uucore's localization setup: true runs about 451k guest instructions, and only about 11.5k of them are its own work. Skipping the setup cuts a call to 0.14 ms but breaks --help and some error messages, so the setup stays. The follow-up is to pre-parse the Fluent bundles in a Wizer snapshot.

Compute is 40-50x slower than native on Pulley. Native builtins therefore stay the default.

Risk

  • Low: off by default. The CPython host refactor is the main shared-code risk. The CPython suites pass unchanged after the hardening: cargo test -p bashkit --features cpython,http_client --lib --test integration -- cpython wasi_host gave 18 lib + 137 integration tests.
  • Security model: TM-WCU-001..008 (see above).
  • Adds a ~2.0 MB published crate (bashkit-coreutils-wasm) and a publish job that bashkit depends on.

Checklist

  • 37 integration tests: registration, multicall, VFS, pipelines, exit codes, env, host isolation and traversal, symlinks, every limit under attack (timeout, cancel, work budget, memory, output, FS limits, path length, concurrency), filter, clock, and a leak sweep over every utility
  • Unit tests: gap list, trap/timeout messages, deadline and cancel waits, errno mapping, tenant share, script-analysis wrapper
  • cargo fmt, clippy clean (default, cpython, wasm-coreutils, combined)
  • just check-okf, check-doc-links, doc routes, scripts unittests
  • Knowledge, threat model, limitations and docs updated
  • Bench baseline saved under crates/bashkit/benches/results/

…ils)

New opt-in `wasm-coreutils` feature: uutils/coreutils 0.12 compiled to
wasm32-wasip1 as one multicall guest (new `bashkit-coreutils-wasm` crate,
71 utilities, precompiled to Pulley at build time), run as sandboxed
builtins over the VFS. Registers `coreutils <util>` plus the utilities
with no native builtin; `wasm_coreutils_replace_native` routes every
guest utility to uutils.

The CPython WASI host moves to `builtins/wasi_host/` and is shared:
optional read-only overlay, HTTP state only with `cpython`, one run loop
for pool slots, deadline/budget polling, exit and trap mapping.

Adds integration/security tests, a criterion bench + startup example
with saved results, knowledge (runtimes/wasm-coreutils, TM-WCU-*,
L-WCU-*), rustdoc guide, CI and publish wiring.
@chaliy chaliy self-assigned this Oct 9, 2026
@cloudflare-workers-and-pages

cloudflare-workers-and-pages Bot commented Oct 9, 2026 •

Copy link
Copy Markdown

Deploying with  Cloudflare Workers  Cloudflare Workers

The latest updates on your project. Learn more about integrating Git with Workers.

Status Name Latest Commit Preview URL Updated (UTC)
✅ Deployment successful!
View logs
bashkit 9f12889 Commit Preview URL

Branch Preview URL
Nov 03 2026, 05:45 AM

chaliy added 8 commits October 9, 2026 03:00
- Enforce the request work budget mid-run: guest fuel starts at the
  remaining work units, so a long call traps instead of being charged after.
- Re-check deadline and cancellation every 50 ms while waiting for a pool
  slot or tenant share (unwoken waits ran past the deadline).
- Bound host copies of guest data before making them: fd_write gathers only
  what the output cap or file size limit admits, oversized paths are refused
  unread, directory listings are cached and charged, /dev/null is a sink.
- Map FS limit errors to EFBIG/ENOSPC/ENAMETOOLONG and loops to ELOOP.
- Apply builtin_filter to guest utilities and the multicall; embedder
  builtins win; report `coreutils` as a script-analysis command wrapper.
- Serve the virtual clock to guests (wall time and file mtimes), coarse
  under the Hardened profile; per-call monotonic time.
- Cap guests per Bash with max_concurrent (default 4) before the shared pool.
- Mask exit status to 8 bits; release reservations on fd exhaustion.
- Guest: panic hook prints `<util>: internal error`; drop dd (thread spawn
  traps on wasip1).

@bashkit-pr-reviewer bashkit-pr-reviewer Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reviewed the wasm boundary (wasi_host move, wasm_coreutils runner, guest build, analysis/threat-model updates). Two findings below; everything else I probed (fuel/deadline metering, exit masking, env scoping, filter checks, embedder-wins install order, output/file caps) holds up.

let from = c.data().resolve(fd1 as u32, &old_raw)?;
let to = c.data().resolve(fd2 as u32, &new_raw)?;
if c.data().is_stdlib(&from) || c.data().is_stdlib(&to) {
if c.data().is_overlay(&from) || c.data().is_overlay(&to) {

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[P1] path_rename only blocks the exact overlay path, so renaming an overlay parent escapes the read-only overlay. E.g. with overlay at /data/in.txt, mv /data /other passes this check (from=/data != overlay) and moves the directory; afterwards /other/in.txt is an ordinary writable file (stale overlay_path still points at /data/in.txt) and the guest can overwrite formerly read-only bytes. Evidence: wasi.rs:1331 checks is_overlay(&from) || is_overlay(&to), while path_remove_directory (~1290) and path_create_directory check is_overlay_ancestor(&p) too. Fix: reject rename when from/to is the overlay or an ancestor of it (and when the overlay is an ancestor of from/to, i.e. either side is == overlay, an ancestor of the overlay, or has the overlay as an ancestor). Same gap exists in path_filestat_set_times (exact-only check) for mtime of overlay parents — worth the same treatment.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Traced this and it does not reach the read-only bytes. The overlay is never a VFS file: GuestState.overlay holds the embedded &'static [u8], and path_open (~1693) and filestat (~695) serve it purely by comparing the resolved path with the fixed overlay_path. Renaming an ancestor directory moves only that directory's VFS entries. The overlay keeps answering at its original path, and /other/in.txt would not exist with overlay contents, so nothing writable ever carries those bytes. The same applies to path_filestat_set_times on an ancestor: it touches only a VFS directory's mtime. Only the cpython guest has an overlay (the stdlib zip); wasm-coreutils passes none. Leaving this as is: a guard here would add code without protecting anything.


Generated by Claude Code

@bashkit-pr-reviewer

Copy link
Copy Markdown

Review summary (head c8abc7a)

Verdict: solid, carefully-isolated change — one P1 sandbox-integrity gap plus one P2 robustness issue (both filed as inline comments). No blockers beyond the P1.

Findings

  • P1: path_rename checks only the exact overlay path, so renaming an overlay parent (e.g. mv /data /other with overlay at /data/in.txt) moves read-only data into an ordinary writable file. Sibling handlers (path_remove_directory, path_create_directory) already use is_overlay_ancestor; rename (and path_filestat_set_times) do not.
  • P2: path_symlink accepts empty and NUL-containing targets unchecked.

What I checked beyond the diff: wasi_host/mod.rs (fuel/deadline/budget metering, pool sizing, exit/trap mapping incl. 8-bit masking in proc_exit), wasi.rs path resolution/clamping, open/read/write/flush paths, fd_write truncation accounting, wasm_coreutils.rs (argv/env validation, filter checks on both direct and coreutils <util> dispatch — nested coreutils coreutils … does not bypass since argv[0] must be a utility, embedder-wins or_insert install order, truncation surfacing), bashkit-coreutils-wasm/build.rs + guest/ build, analysis.rs wrapper treatment (consistent with existing wrappers), AGENTS.md + knowledge/runtimes/wasm-coreutils.md + threat-model updates (TM-WCU coverage matches the code).

Not reviewed: truncated was false for the diff call I made; I read all changed areas from the checkout instead, so nothing was skipped for lack of visibility. I did not build or run tests per the sandbox rules — findings are from code reading.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant