diff --git a/.claude/knowledge/blake3-on-ndarray-simd.md b/.claude/knowledge/blake3-on-ndarray-simd.md new file mode 100644 index 00000000..c38735c7 --- /dev/null +++ b/.claude/knowledge/blake3-on-ndarray-simd.md @@ -0,0 +1,213 @@ +# BLAKE3 without C and without intrinsics — what is actually missing + +> **Status: MEASURED, 2026-07-29.** Every number below comes from +> `.claude/knowledge/simd-codegen-oracle/` on x86_64 @ `x86-64-v3`, +> rustc 1.95.0. The intrinsic census comes from `AdaWorldAPI/BLAKE3` +> at `8aa5145`. + +## READ BY: +- Anyone porting BLAKE3 onto `ndarray::simd` +- Anyone about to hand-write shuffle intrinsics for a lane type +- `simd-savant`, `truth-architect` + +## P0 TRIGGER +About to write `_mm256_unpacklo_epi32` wrappers because `U32x16` has no +shuffle surface? **Don't. Measured below: the interleave primitive compiles +to a real packed permute from a plain scalar index loop. What is missing is +a method, not an intrinsic.** + +--- + +## The starting position + +Two constraints, both operator-set: + +1. **No FFI with C.** `blake3` is now pinned to `default-features = false, + features = ["pure"]`, which removed 33 `.o` objects and + `libblake3_avx512_assembly.a` from the build. Measured, and already + shipped. +2. **No second SIMD surface.** `pure` gets rid of the C, but its Rust + backends (`rust_avx2.rs`, `rust_sse41.rs`, `rust_sse2.rs`, + `wasm32_simd.rs` — 2,910 lines) are raw `core::arch` intrinsics. That is + exactly what the matryoshka pattern exists to prevent: all SIMD lives + once, audited, inside `ndarray::simd`. + +So the question is what a `ndarray::simd`-native BLAKE3 backend would need. + +## The census: 15 intrinsics, and 12 already exist + +`src/rust_avx2.rs` (496 lines) uses fifteen distinct intrinsics. Grouped by +whether `crate::simd::U32x16` can already express them: + +| intrinsic | uses | `U32x16` today | +|---|---|---| +| `_mm256_add_epi32` | 1 | `Add` | +| `_mm256_xor_si256` | 1 | `BitXor` | +| `_mm256_or_si256` | 4 | `BitOr` | +| `_mm256_slli_epi32` / `_mm256_srli_epi32` | 4 / 4 | folded into `rotate_left` | +| `_mm256_set1_epi32` | 1 | `splat` | +| `_mm256_setr_epi32` / `_mm256_set_epi32` | 1 / 1 | `from_array` | +| `_mm256_loadu_si256` / `_mm256_storeu_si256` | 1 / 1 | `from_slice` / `copy_to_slice` | +| **`_mm256_unpacklo_epi32` / `_mm256_unpackhi_epi32`** | **4 / 4** | **absent** | +| **`_mm256_unpacklo_epi64` / `_mm256_unpackhi_epi64`** | **4 / 4** | **absent** | +| **`_mm256_permute2x128_si256`** | **2** | **absent** | + +The twelve available ones cover the whole compression core. The three +missing families — 18 call sites — exist for one thing: the transpose in +`hash_many`, which turns N chunk states into word-major vectors so the +rounds run N ways in parallel. + +`U32x16` is generated by `avx2_int_type!` and exposes `splat`, `from_slice`, +`from_array`, `to_array`, `copy_to_slice`, `reduce_sum`, the operators, and +`rotate_left`. There is no shuffle surface at all. + +## Four measurements + +The tempting move is to hand-write the missing shuffles as intrinsics. TD-T22 +is the standing reason not to assume that: a 700-line intrinsic PR was closed +after measurement showed LLVM was already at the instruction floor. So the +shuffles got measured first. + +| probe | packed | scalar (lane data) | what LLVM emitted | +|---|---|---|---| +| `blake3_g_u32x16` | **72** | 0 | `vpshufb` ×2 (rotr 16, rotr 8), `vpsrld`+`vpslld`+`vpor` ×2 (rotr 12, rotr 7), `vpaddd`/`vpxor` throughout | +| `interleave_lo_u32x16` | **11** | 0 | `vbroadcasti128` + `vpmovzxdq` + `vpermd` (constant mask) + `vpblendd $170` | +| `transpose_16x16_u32` | **0** | 1 | 1088 B of stack, a 256-iteration scalar element copy, stride-64 scatter | +| `transpose_stage_u32x16` | **27** | 1 | `vunpcklps` / `vunpckhps` — one 32-bit stage only | +| `transpose_16x16_composed` | **79** | 0 | 60 vector moves + **19 shuffles**: `vinserti128` ×7, `vpshufd` ×3, `vpblendd` ×3, `vpunpcklqdq` ×2, `vpermq` ×2, `vunpcklpd`, `vpermpd` | + +Rows 3 and 5 are the same transpose, correctness-checked to be the same +function. The difference is only how it is written. + +### The G function is free + +72 packed instructions, zero scalar on lane data. BLAKE3 specifies *right* +rotations by 16/12/8/7 and this crate has no `rotate_right` at any width, so +each is written `rotate_left(32 - n)` — exact, since rotation is modular. The +resulting amounts split as the u32 lane always does: byte-granular 16 and 24 +fold to `vpshufb`, non-byte-granular 20 and 25 take the shift-or triple. + +**`compress_in_place` and `compress_xof` need nothing new.** That half of the +port is a transcription. + +### The interleave primitive is free + +11 packed, zero scalar, from a plain `for i in 0..8 { out[2i]=a[i]; +out[2i+1]=b[i] }`. LLVM synthesized a two-source cross-lane permutation — +`vpermd` against a constant mask, then `vpblendd`. + +This extends `cross_lane_reverse_u8x64`'s earlier result (a *single*-source +permute synthesized from a scalar reverse loop) to the two-source case, which +was the open half. Both were written as index loops; both came out as real +shuffles. + +### The monolithic transpose is not free + +Zero packed. LLVM allocated 1088 bytes of stack (`subq $1088,%rsp`, 64-byte +aligned) and emitted a genuine 256-iteration element-at-a-time copy — +`cmpq $256,%rax` / `jne` — striding the output by 64 bytes each step. + +The stride-64 scatter is what defeats it: each output vector gathers one +element from each of sixteen inputs. Nothing in the loop looks like a +shuffle to the vectorizer. + +### …but the composed transpose is + +A complete, correctness-checked 16×16 transpose, written as four butterfly +stages at granularities 1 / 2 / 4 / 8 (32-bit unpack, 64-bit unpack, 128-bit +lane exchange, 256-bit half exchange) over a `const G` parameter so every +shuffle pattern is compile-time constant: **79 packed, zero scalar on lane +data**, of which 19 are real shuffles — `vinserti128`, `vpshufd`, +`vpblendd`, `vpunpcklqdq`, `vpermq`, `vunpcklpd`, `vpermpd`. All four +granularities appear in the emitted code, including the 64-bit unpack and the +256-bit cross-lane permute. + +The single-stage probe (`transpose_stage_u32x16`, 27 packed, `vunpcklps` / +`vunpckhps`) remains in the oracle as a narrower data point, but it is **not** +what supports the conclusion. It is one 32-bit stage, exercises neither wider +exchange, and its whole-vector helper semantics do not compose into a real +transpose — so an earlier version of this document that rested the conclusion +on it was inferring, not measuring. Raised by codex on PR #265; the finding +was correct, and `transpose_16x16_composed` is the answer to it. + +**Correctness is checked, not asserted.** The driver compares the composed +result against a naive nested-loop transpose and aborts on mismatch, and +`run.sh` now executes the probe binary — `--emit asm` links nothing, so the +assertion would otherwise never run. The check was control-tested by deleting +`stage::<8>`, which makes it fire (exit 101). A packed-but-wrong network +cannot be reported here as a success. + +**Two honest caveats.** Unlike the single stage, this is not clean +straight-line code: LLVM kept loop structure (38 loop-control instructions) +and there is real spill traffic (41 memory). That is expected — sixteen +512-bit vectors of live state are 32 ymm registers against 16 architectural +ones, and a hand-written intrinsic backend faces exactly the same pressure. +So the measurement establishes that the shuffle network is *synthesized* +rather than degraded to scalar copies. It does not establish throughput +parity with `rust_avx2.rs`. + +## What this means + +**No hand-written intrinsic is earned.** Under the entry criterion in +`simd-one-spec-design.md`, an override needs a probe showing the generic form +fails. The generic form does not fail here — the primitive vectorizes, and so +does a composition of primitives. Only the monolithic index-loop spelling +fails, and that is a spelling, not a capability. + +So what is missing is a **method surface on `U32x16`**, implemented as the +scalar index loops LLVM already handles. The measured probe suggests it wants +to be *one* const-generic operation rather than five named ones: + +```rust +fn exchange(lo: Self, hi: Self) -> (Self, Self) +``` + +`G = 1` is the 32-bit unpack, `G = 2` the 64-bit unpack, `G = 4` the 128-bit +lane exchange, `G = 8` the 256-bit half exchange: + +| `G` | intrinsic role | replaces in `rust_avx2.rs` | +|---|---|---| +| 1 | 32-bit unpack | `_mm256_unpack{lo,hi}_epi32` (8 sites) | +| 2 | 64-bit unpack | `_mm256_unpack{lo,hi}_epi64` (8 sites) | +| 4 | 128-bit lane exchange | `_mm256_permute2x128_si256` (2 sites) | +| 8 | 256-bit half exchange | (no AVX2 analogue — a 512-bit lane needs it) | + +About fifteen lines, once. **Zero `unsafe`, zero `core::arch`, zero C.** The +transpose is a composition of four stages over it — which is what the +intrinsic backends already are, spelled in the lane vocabulary instead of in +x86. The measured `transpose_16x16_composed` probe is that composition, +verified to be a correct transpose. + +That is the whole gap. It is a method surface, not a porting effort. + +## Deliberate scope limits + +- **Degree 16, not 8.** `U32x16` is the lane the substrate uses, so the + natural backend matches BLAKE3's AVX-512 degree rather than AVX2's degree 8. + Degree 8 would want a `U32x8`, ruled out as a building block (operator + ruling, 2026-07-28). +- **`transpose_stage_u32x16` measures codegen shape, not correctness.** Its + helpers use whole-vector interleave semantics rather than x86's + per-128-bit-lane `unpack`, so four stages of *those* helpers do not compose + a correct transpose. Getting the lane semantics right is a detail of the + real implementation; the question the probe answers is whether composing + interleave calls stays packed, and it does. +- **Nothing here is measured on aarch64 or wasm32.** The oracle needs a + baseline per target and only `baseline-x86_64-v3.toml` exists. +- **No benchmark.** Every claim above is about *instruction class*, not + throughput. "Emits packed shuffles" is not "is faster than the intrinsic + backend" — that needs a bench against `rust_avx2.rs`, which has not been + run. + +## Still open + +1. **Does the composed transpose beat `rust_avx2.rs`?** Unmeasured on both + sides. The instruction-class result says a port is *possible* without + intrinsics; it does not say it wins. +2. **The FIPS gate.** If any deployment needs FIPS-adjacent claims, BLAKE3 is + out and SHA-384 stays the KDF hash (`crypto-lane-status.md`). Settle + before investing in the port. +3. **`hash_many`'s degree contract.** BLAKE3's `Platform::simd_degree()` + feeds chunk scheduling. A degree-16 backend on an AVX2 host is legal but + changes the parallelism/transpose-cost balance — worth measuring, not + assuming. diff --git a/.claude/knowledge/chacha20-vendoring-blast-radius.md b/.claude/knowledge/chacha20-vendoring-blast-radius.md new file mode 100644 index 00000000..cfdb90e9 --- /dev/null +++ b/.claude/knowledge/chacha20-vendoring-blast-radius.md @@ -0,0 +1,181 @@ +# `vendor/chacha20` — what it actually reaches + +> **Status: MEASURED, 2026-07-29.** Every row below is read from `Cargo.lock`, +> a manifest on disk, `rustc --print cfg`, or a git object hash. No estimates. + +## READ BY: +- Anyone about to re-sync, extend, or delete `vendor/chacha20` +- Anyone citing "the AdaWorldAPI chacha20 fork" as an accelerated dependency +- Anyone applying the P0 fork rule to a crate that also has a vendored copy + +## P0 TRIGGER +About to state that a consumer's ChaCha20 keystream is accelerated because +ndarray patches `chacha20`? **Read the two reach tables first. The patch and +the acceleration have different, and both narrower-than-assumed, reach.** + +--- + +## Two different questions, two different answers + +"Does the fork reach consumer X?" is really two questions, and conflating +them is what produced the overstated claims in the manifest comments. + +1. **Does the `[patch.crates-io]` apply** — i.e. does the build link the + vendored source at all? +2. **Does `ndarray_simd` compile** — i.e. does the added backend get + selected, or does the vendored crate fall through to RustCrypto's own? + +A build can answer yes to (1) and no to (2). Most do. + +## Reach of the patch + +`[patch.crates-io]` takes effect **only from the top-level workspace +manifest**. It is ignored in a dependency's manifest, so ndarray's patch +does not travel with ndarray — each consumer must declare its own. + +| repo | declares a chacha20 patch | links the vendored source | +|---|---|---| +| ndarray | yes — `Cargo.toml:488` | **yes** | +| MedCare-rs | yes — `vendor/ndarray/vendor/chacha20` | **yes** | +| lance-graph | no | no — registry crate | +| OGAR | no | no — registry crate | +| tesseract-rs | no | no — registry crate | +| stockfish-rs | no | no — registry crate | +| a2ui-rs | no | no — registry crate | + +Evidence for the two "yes" rows: ndarray's `Cargo.lock` carries +`chacha20 0.9.1` with **no `source` line**, which is how a path override +appears; MedCare-rs's root manifest declares +`chacha20 = { path = "vendor/ndarray/vendor/chacha20" }`, and its +`vendor/ndarray` symlink canonicalizes to the same checkout, so one +`ndarray` node resolves. + +Evidence for the five "no" rows: `grep 'chacha20'` and `grep '\[patch'` +across each root manifest. lance-graph's manifest contains only a comment +*about* patching, explicitly recording that no `[patch]` block exists there. + +## Reach of the `ndarray_simd` backend + +`vendor/chacha20/src/backends.rs` selects the added backend under + +```rust +all(target_arch = "x86_64", target_feature = "avx512f", …) // or +all(target_arch = "wasm32", target_feature = "simd128") +``` + +`avx512f` is a **compile-time** feature cfg, so it is present only if the +build pins a target-cpu that includes it. Measured: + +```console +$ rustc --print cfg -Ctarget-cpu=x86-64-v3 | grep avx +target_feature="avx2" # <- no avx512f +$ rustc --print cfg -Ctarget-cpu=x86-64-v4 | grep avx512f +target_feature="avx512f" +``` + +| build | target-cpu | `avx512f` | backend compiled | +|---|---|---|---| +| ndarray, default | `x86-64-v3` (`.cargo/config.toml`) | absent | RustCrypto soft/avx2/sse2 | +| ndarray, `--config .cargo/config-avx512.toml` | `x86-64-v4` | present | **`ndarray_simd`** | +| MedCare-rs, default | none — no `.cargo/config.toml` at all | absent | RustCrypto soft/avx2/sse2 | +| wasm32 + `simd128` | — | n/a | **`ndarray_simd`** | +| the five unpatched repos | — | — | not present in the source they link | + +**No default build of any repo in the workspace runs `ndarray_simd`.** Two +opt-in configurations do: an explicitly-selected AVX-512 x86_64 build, and a +wasm32 build with `simd128`. + +This is not a defect — the fallthrough is correct and the comments in +MedCare-rs's manifest describe it accurately ("non-avx512 builds fall through +to RustCrypto's own backends"). It is a scope fact that keeps being stated +one size too large. + +## One manifest comment overstates it + +`Cargo.toml:481-482` reads: + +> This transitively accelerates the `encryption` crate's XChaCha20-Poly1305 +> keystream … **on any x86_64+avx512f build (the workspace's +> `target-cpu=x86-64-v4`)**. + +The workspace's pinned target-cpu is **v3**, in `.cargo/config.toml`. `v4` +is the opt-in `.cargo/config-avx512.toml`. The conditional half of the +sentence is right; the parenthetical asserts the default build satisfies it, +and it does not. + +**Not edited here.** Manifest string changes in this repo need an operator +ruling; the finding is recorded instead of applied. + +## The fork the P0 rule names is not this tree + +`AdaWorldAPI/stream-ciphers` exists and is now cloned in-session. + +| | `AdaWorldAPI/stream-ciphers` | `ndarray/vendor/chacha20` | +|---|---|---| +| what it is | mirror of RustCrypto upstream | hand-copied source tree | +| head | `5f3430b` "Release chacha20 v0.10.1 (#574)" | — | +| branches | `master` only | — | +| chacha20 version | **0.10.1** | **0.9.1** | +| `cipher` | 0.5 | 0.4.4 | +| edition / rust-version | 2024 / 1.85 | 2021 / 1.95 | +| `backends/ndarray_simd.rs` | **absent** | present | +| `backends/avx512.rs` | **present (upstream's own)** | absent | +| `repository` field | RustCrypto | rewritten to `AdaWorldAPI/ndarray` | + +Three consequences worth stating plainly: + +1. **The fork carries no AdaWorldAPI delta**, and this is checkable rather + than inferred from a branch count. Against `RustCrypto/stream-ciphers` + fetched as `upstream`: + + ```console + $ git rev-parse HEAD + 5f3430b7531a33aa14957b6bd407b46687635124 + $ git rev-parse upstream/master + 5f3430b7531a33aa14957b6bd407b46687635124 + $ git rev-list --count upstream/master..HEAD + 0 + $ git rev-parse HEAD:chacha20 upstream/master:chacha20 + 3f58304be34b79a3a38430a5752fb8db223d1a20 + 3f58304be34b79a3a38430a5752fb8db223d1a20 + ``` + + The fork's head *is* upstream's head — same commit hash, zero commits + ahead, and the `chacha20/` subtree hashes to the same object. Git tree + hashes cover content recursively, so identical tree ids mean byte equality + for that directory. "Depend on the fork" and "depend on upstream" are + therefore the same bytes today — as a verified fact, not an inference. + (Repeat the check after any upstream sync; it is one command.) +2. **The vendored tree is not a checkout of the fork**, and is one minor + release behind it. It cannot be re-synced by `git pull`; the re-sync + procedure its own header describes (bump the source, re-apply the backend + and the four cfg branches, re-run the vectors) is a manual port, and + 0.9.1 → 0.10.1 crosses a `cipher` major (0.4 → 0.5). +3. **Upstream now ships its own AVX-512 backend.** `backends/avx512.rs` + at 0.10.1 occupies the exact niche `ndarray_simd.rs` was written to fill. + Whether the ndarray-lane version still earns its place is a measurement + question — and the instrument for it already exists + (`.claude/knowledge/simd-codegen-oracle/`), with the relevant lane already + measured: the u32 ARX triple on `U32x16` hits the AVX2 instruction floor + (`td-t22-asm-investigation.md`). + +## What is NOT claimed here + +- Not that the vendored backend is wrong. It is untested by default builds, + which is a different statement. +- Not that the patch should be removed. That is a decision, and it depends on + (3) above plus the AVX-512 deployment question, neither of which this audit + settles. +- Not that MedCare-rs's wiring is incorrect. It is the one consumer that + declared its patch deliberately and documented the fallthrough honestly. + +## Open, for an operator ruling + +1. **Point `vendor/chacha20` at the fork, or keep the copy?** The P0 rule + says depend on the AdaWorldAPI fork. Today the tree is neither — a copy of + a version the fork does not carry, with `repository` rewritten to point at + ndarray itself. +2. **Does `ndarray_simd` still beat upstream's own `avx512.rs`?** Answerable + with the oracle. Unmeasured today, on both sides. +3. **The v3/v4 manifest comment** — correct in place, or leave the record and + annotate? diff --git a/.claude/knowledge/simd-codegen-oracle/README.md b/.claude/knowledge/simd-codegen-oracle/README.md index 206e2a4e..d5b58545 100644 --- a/.claude/knowledge/simd-codegen-oracle/README.md +++ b/.claude/knowledge/simd-codegen-oracle/README.md @@ -28,32 +28,18 @@ prevent. Measure during design instead. ## Running it -The tool needs a throwaway crate to compile the probes against `ndarray`: - ```sh -cd /path/to/ndarray -mkdir -p /tmp/oracle/src -cp .claude/knowledge/simd-codegen-oracle/probes.rs /tmp/oracle/src/main.rs -cat > /tmp/oracle/Cargo.toml <<'EOF' -[package] -name = "simd-codegen-oracle" -version = "0.0.0" -edition = "2021" -[dependencies] -ndarray = { path = "/path/to/ndarray", features = ["std"] } -[profile.release] -debug = false -EOF - -cargo rustc --release --manifest-path /tmp/oracle/Cargo.toml -- \ - --emit asm -C debuginfo=0 -C target-cpu=x86-64-v3 -python3 .claude/knowledge/simd-codegen-oracle/analyze.py \ - "$(ls -t /tmp/oracle/target/release/deps/simd_codegen_oracle-*.s | head -1)" \ - .claude/knowledge/simd-codegen-oracle/baseline-x86_64-v3.toml +sh .claude/knowledge/simd-codegen-oracle/run.sh # host, x86-64-v3 baseline +sh .claude/knowledge/simd-codegen-oracle/run.sh # cross-target +sh .claude/knowledge/simd-codegen-oracle/run.sh --verbose # per-probe instruction detail ``` -`run.sh` automates this against a crate laid out as above; adjust -`MANIFEST`/`BASELINE` at the top for your scratch location. +No setup: the script locates the ndarray checkout it lives in, builds a +throwaway crate from `probes.rs` under `$TMPDIR`, emits assembly, and hands +the `.s` to `analyze.py`. The scratch tree is removed on exit. + +Adding a target means adding a `baseline-.toml`; the script refuses +to guess and exits 90 if one is missing. ## Two properties that make the result trustworthy diff --git a/.claude/knowledge/simd-codegen-oracle/baseline-x86_64-v3.toml b/.claude/knowledge/simd-codegen-oracle/baseline-x86_64-v3.toml index 22f0b8c0..975d3b59 100644 --- a/.claude/knowledge/simd-codegen-oracle/baseline-x86_64-v3.toml +++ b/.claude/knowledge/simd-codegen-oracle/baseline-x86_64-v3.toml @@ -120,3 +120,39 @@ note = "Same pattern as rot_u64x8, 4 lanes instead of 8, same verdict. Observed [probe.blake2b_g_u64x8] expect = "unknown" note = "MEASURED MIXED, and the most interesting single result in this oracle. The G-function's straight-line body has NO loop at all (fully unrolled by construction -- 8 lanes, no trip count, confirmed by 0 jumps in the disassembly). The LEADING a=a+b step DOES vectorize (vpaddq over both 32-byte ymm halves -- 2 of the function's 22 packed instructions; the rest of the 22 are the final vmovaps/vmovups reassembly into the return-struct). The moment a rotate is needed, LLVM extracts every lane out to a GPR (vmovq/vpextrq, in the 'other'/memory buckets) and the REST of the function -- all four rotate stages (32/24/16/63) and essentially every subsequent add/xor -- stays scalar (rorxq/addq/xorq on GPRs) all the way through, only reassembling into ymm registers at the very end for the tuple-return store. Byte-granular rotate amounts (32/24/16 -- whole bytes, the exact shape that folds to vpshufb for u32's rotate(16)/(8)) did NOT fold to a shuffle here either -- all four rotate amounts, byte-granular or not, lowered to scalar rorxq identically. This directly answers the motivating question: no, u64 rotate does not get the u32 lane's free ride, not even for the byte-granular special case, not even with compile-time-constant rotate amounts (unlike rot_u64x8/x4's runtime-variable case, ruling out 'maybe it only needed a constant amount' as an explanation). Observed 22 packed (the leading vpaddq pair + the trailing reassembly moves) / 74 scalar-lane-arith (rorxq/addq/xorq on GPRs, automated count) / 14 loop-control (automated count) / 98 memory (lane extract/insert + loads/stores) / 2 other. KNOWN UNDER-COUNT: the 14 instructions the automated classifier placed in loop-control are, on manual inspection, ALSO genuine scalar lane arithmetic (xorq/addq on %r8/%rsi) -- they were misclassified because those two registers were used as memory-operand address bases earlier in the function (dereferencing the incoming d-pointer) before being overwritten and reused to carry rotated lane data, and the whole-block heuristic in scripts/codegen_oracle_analyze.py cannot distinguish a register's role before vs. after such a reuse (documented in that script's module docstring, 'KNOWN LIMITATION' paragraph). The true scalar-lane-arith count is therefore ~88, not 74; the qualitative verdict (packed only for the leading add and the trailing reassembly; scalar for the entire rotate-dependent chain in between) is unaffected either way and was cross-checked by hand against the raw --verbose disassembly." + +# ============================================================================ +# Group D -- UNKNOWN. Report only, no pass/fail. Does a fixed lane permutation +# written as a scalar index loop vectorize? This is the one thing standing +# between `ndarray::simd::U32x16` and a BLAKE3 backend with no raw intrinsics +# and no C: BLAKE3's pure-Rust AVX2 backend uses 15 distinct intrinsics, 12 of +# which U32x16 already expresses, and the other 3 families (unpacklo/hi_epi32, +# unpacklo/hi_epi64, permute2x128 -- 18 call sites) exist solely to build +# hash_many's transpose. +# +# `blake3_g_u32x16` carries no suspense and is here as the anchor: the same +# ARX shape Group A already measured at the AVX2 instruction floor. It is +# marked "unknown" rather than "vectorized" only so the Group-D block reads as +# one measurement rather than a pass mixed with two observations; if it ever +# stops vectorizing, arx_rounds_u32x16 fails first and louder. +# ============================================================================ + +[probe.blake3_g_u32x16] +expect = "unknown" +note = "MEASURED FULLY VECTORIZED, as predicted. Observed 72 packed / 0 scalar-lane-arith / 0 loop-control (straight-line, fully unrolled) / 1 memory. The rotate split came out exactly as the u32 lane's known behaviour implies: rotr16 -> vpshufb with mask .LCPI5_0, rotr8 -> vpshufb with mask .LCPI5_1, rotr12 -> vpsrld $12 + vpslld $20 + vpor, rotr7 -> the same triple. Everything else is vpaddd / vpxor over both ymm halves. BLAKE3's compression core therefore needs NOTHING new from this crate -- compress_in_place and compress_xof are expressible on U32x16 today." + +[probe.interleave_lo_u32x16] +expect = "unknown" +note = "MEASURED FULLY VECTORIZED. Observed 11 packed / 0 scalar-lane-arith / 0 loop-control. LLVM emitted vbroadcasti128 + vpmovzxdq + vpermd (constant permute mask from .LCPI10_0) + vpblendd $170 -- a genuine two-source cross-lane permutation synthesized from index arithmetic, extending cross_lane_reverse_u8x64's single-source result to the two-source case. The interleave primitive is FREE from scalar source; it earns no intrinsic override." + +[probe.transpose_16x16_u32] +expect = "unknown" +note = "MEASURED SCALAR -- and this is the finding. Observed 0 packed / 1 scalar-lane-arith / 8 loop-control / 47 memory. LLVM did not vectorize any part of it: it allocated 1088 bytes of stack (subq $1088,%rsp with a 64-byte alignment andq $-64,%rsp) and emitted a real 256-iteration element-at-a-time copy loop (cmpq $256,%rax / jne), striding output by 64 bytes per step. The stride-64 scatter is what defeats it -- each output vector gathers one element from each of sixteen inputs. Compare transpose_stage_u32x16 below, which is the SAME transpose expressed as a composition of interleaves and stays fully packed. The lesson is about how the transpose is WRITTEN, not about what the hardware or the lane type can do." + +[probe.transpose_stage_u32x16] +expect = "unknown" +note = "MEASURED FULLY VECTORIZED, and it closes the Group-D question. Observed 27 packed / 1 scalar-lane-arith / 5 loop-control / 9 memory. LLVM emitted vunpcklps / vunpckhps -- literally the unpack instruction family BLAKE3's rust_avx2.rs hand-writes as _mm256_unpacklo_epi32 / _mm256_unpackhi_epi32 -- four pairs per loop iteration over a 1024-byte stride. The single scalar-lane-arith is an xorl %esi,%esi zero-idiom from the [U32x16::splat(0); 16] initializer, not lane work. Read against transpose_16x16_u32's 0 packed: the transpose must be COMPOSED from interleave primitives rather than written as a nested index loop. Since the primitive itself vectorizes from scalar source (interleave_lo_u32x16), the conclusion is that a BLAKE3 backend on ndarray::simd needs no hand-written intrinsic at all -- only that the shuffle methods EXIST on U32x16." + +[probe.transpose_16x16_composed] +expect = "unknown" +note = "MEASURED VECTORIZED, with a genuine shuffle network. Observed 79 packed / 0 scalar-lane-arith / 38 loop-control / 41 memory. The 79 packed split into 60 vector moves and 19 real shuffle/permute instructions: vinserti128 x7, vpshufd x3, vpblendd x3, vpunpcklqdq x2, vpermq x2, vunpcklpd x1, vpermpd x1 -- covering all four granularities the network needs, including the 64-bit unpack and the 256-bit cross-lane permute that transpose_stage_u32x16 never exercised. Correctness is CHECKED, not assumed: the driver compares the result against a naive nested-loop transpose and aborts on mismatch, and run.sh now executes the probe binary (--emit asm links nothing, so the assertion would otherwise never run). The check was control-tested by deleting stage::<8>, which makes it fire (exit 101). Honest caveats: unlike the single-stage probe this is NOT clean straight-line code -- LLVM kept loop structure (38 loop-control) and there is real spill traffic (41 memory), which is expected, since sixteen 512-bit vectors of live state are 32 ymm registers against 16 architectural ones. A hand-written intrinsic backend faces the same register pressure. So this measurement establishes that the shuffle network is SYNTHESIZED rather than degraded to scalar copies; it does NOT establish throughput parity with rust_avx2.rs, which needs a benchmark that has not been run." diff --git a/.claude/knowledge/simd-codegen-oracle/probes.rs b/.claude/knowledge/simd-codegen-oracle/probes.rs index d8e9b63b..f26807b5 100644 --- a/.claude/knowledge/simd-codegen-oracle/probes.rs +++ b/.claude/knowledge/simd-codegen-oracle/probes.rs @@ -263,6 +263,236 @@ pub fn blake2b_g_u64x8(a: U64x8, b: U64x8, c: U64x8, d: U64x8) -> (U64x8, U64x8, (a, b, c, d) } +// ============================================================================ +// Group D — UNKNOWN. What a BLAKE3 backend on `ndarray::simd` would need. +// ============================================================================ +// BLAKE3's pure-Rust AVX2 backend (`AdaWorldAPI/BLAKE3`, `src/rust_avx2.rs`, +// 496 lines) uses exactly 15 distinct intrinsics. Twelve of them are already +// expressible on `crate::simd::U32x16` today — add / xor / or / shift / splat +// / load / store / set — and TD-T22 measured that lane family at the AVX2 +// instruction floor. The remaining three families are a lane-shuffle network, +// with 18 call sites between them: +// +// _mm256_unpacklo_epi32 / _mm256_unpackhi_epi32 (4 + 4) +// _mm256_unpacklo_epi64 / _mm256_unpackhi_epi64 (4 + 4) +// _mm256_permute2x128_si256 (2) +// +// They exist for one purpose: the transpose in `hash_many`, which turns N +// chunk states into word-major vectors so the compression rounds run N ways +// in parallel. `U32x16` has no shuffle surface at all — the macro-generated +// `avx2_int_type!` types expose splat / from_slice / from_array / to_array / +// copy_to_slice / reduce_sum and the operators, nothing more. +// +// So the open question is NOT "does BLAKE3's mixing vectorize" — Group A +// already answers that for the same ARX shape. It is: **does a fixed lane +// permutation, written as a scalar index loop, get the same free ride?** +// +// There is real evidence on both sides, which is why this is UNKNOWN rather +// than a hypothesis dressed up as an expectation: +// +// FOR — `cross_lane_reverse_u8x64` (Group B) is a scalar index loop over +// 64 bytes, and LLVM emitted `vbroadcasti128` + `vpshufb` + `vpermq`. It +// synthesized a cross-lane permute unprompted. If that generalizes, the +// whole shuffle network is free and a BLAKE3 backend needs no new API. +// +// AGAINST — that probe permutes ONE vector by a pattern expressible as a +// single shuffle. A transpose reads sixteen vectors and writes sixteen +// more; LLVM has to keep 256 values live and recognize the whole network. +// Nothing in this oracle has tested that shape. +// +// Three probes, escalating in difficulty. If `transpose_16x16_u32` +// vectorizes, no shuffle API is needed. If only `interleave_lo_u32x16` does, +// the surface to add is the interleave pair and the transpose composes from +// it. If neither does, the shuffle network is the second earned intrinsic +// override after the u64 rotate. + +/// BLAKE3's G mixing function over `U32x16` — the u32 sibling of +/// `blake2b_g_u64x8`, and the kernel `compress_in_place` / `compress_xof` are +/// built from. +/// +/// BLAKE3 specifies RIGHT rotations by 16 / 12 / 8 / 7. This crate has no +/// `rotate_right` at any width (measured: zero occurrences across all six +/// backends), so each is written as `rotate_left(32 - n)` — exact, not an +/// approximation, since rotation is modular. The resulting left amounts are +/// 16 / 20 / 24 / 25: two byte-granular (16, 24 — the shape that folds to +/// `vpshufb` on this lane) and two not (20, 25 — the shift-or shape). A split +/// between the two in the histogram is expected and is not a defect. +#[inline(never)] +pub fn blake3_g_u32x16( + a: U32x16, + b: U32x16, + c: U32x16, + d: U32x16, + mx: U32x16, + my: U32x16, +) -> (U32x16, U32x16, U32x16, U32x16) { + let (mut a, mut b, mut c, mut d) = (a, b, c, d); + a = a + b + mx; + d = (d ^ a).rotate_left(16); // rotr 16 + c = c + d; + b = (b ^ c).rotate_left(20); // rotr 12 + a = a + b + my; + d = (d ^ a).rotate_left(24); // rotr 8 + c = c + d; + b = (b ^ c).rotate_left(25); // rotr 7 + (a, b, c, d) +} + +/// A fixed two-source lane interleave — the `_mm256_unpacklo_epi32` role, +/// written as a scalar index loop. +/// +/// Semantics are the straightforward whole-vector form (`out[2i] = a[i]`, +/// `out[2i+1] = b[i]` over the low half), NOT x86's per-128-bit-lane +/// `unpacklo`. The difference is deliberate: the question is whether LLVM can +/// synthesize *a* fixed two-source permutation from index arithmetic at all. +/// If it can, matching x86's exact lane-splitting is a detail of how the +/// backend composes it; if it cannot, the exact semantics are moot. +#[inline(never)] +pub fn interleave_lo_u32x16(a: U32x16, b: U32x16) -> U32x16 { + let (aa, bb) = (a.to_array(), b.to_array()); + let mut out = [0u32; 16]; + for i in 0..8 { + out[2 * i] = aa[i]; + out[2 * i + 1] = bb[i]; + } + U32x16::from_array(out) +} + +/// The real question: a full 16x16 `u32` transpose over `[U32x16; 16]`, +/// written as the obvious nested index loop. +/// +/// This is `hash_many`'s transpose at degree 16 — the width `U32x16` implies, +/// matching what BLAKE3's AVX-512 backend does with `__m512i` rather than the +/// degree 8 its AVX2 backend does with `__m256i`. (Degree 8 would want a +/// `U32x8`, ruled out as a building block by operator ruling, 2026-07-28: the +/// lane the substrate uses is 16 wide.) +/// +/// 256 `u32` values move. If LLVM emits a shuffle network here, a BLAKE3 +/// backend on `ndarray::simd` needs no new primitives at all, and the 18 +/// shuffle call sites in `rust_avx2.rs` have no counterpart to port. +#[inline(never)] +pub fn transpose_16x16_u32(m: [U32x16; 16]) -> [U32x16; 16] { + let src: [[u32; 16]; 16] = std::array::from_fn(|i| m[i].to_array()); + let mut dst = [[0u32; 16]; 16]; + for i in 0..16 { + for j in 0..16 { + dst[j][i] = src[i][j]; + } + } + std::array::from_fn(|i| U32x16::from_array(dst[i])) +} + +/// The high sibling of [`interleave_lo_u32x16`], `#[inline(always)]` so it +/// expands into the staged probe below rather than hiding behind a `call`. +#[inline(always)] +fn interleave_hi(a: U32x16, b: U32x16) -> U32x16 { + let (aa, bb) = (a.to_array(), b.to_array()); + let mut out = [0u32; 16]; + for i in 0..8 { + out[2 * i] = aa[8 + i]; + out[2 * i + 1] = bb[8 + i]; + } + U32x16::from_array(out) +} + +/// `#[inline(always)]` twin of the `interleave_lo_u32x16` probe, for use +/// inside the staged transpose below. +#[inline(always)] +fn interleave_lo(a: U32x16, b: U32x16) -> U32x16 { + let (aa, bb) = (a.to_array(), b.to_array()); + let mut out = [0u32; 16]; + for i in 0..8 { + out[2 * i] = aa[i]; + out[2 * i + 1] = bb[i]; + } + U32x16::from_array(out) +} + +/// One butterfly STAGE of a transpose network — eight pairwise +/// interleave-lo/hi over sixteen vectors. Four such stages compose a 16x16 +/// transpose, which is structurally what the `unpacklo`/`unpackhi`/`permute` +/// network in BLAKE3's `rust_avx2.rs` is. +/// +/// **This probe measures codegen shape, not transpose correctness.** Because +/// `interleave_lo`/`_hi` here use whole-vector semantics rather than x86's +/// per-128-bit-lane `unpack`, four stages of *these* helpers do not compose +/// into a correct transpose. That is irrelevant to the question being asked: +/// if a stage built from interleave calls stays packed while the monolithic +/// index-loop transpose does not, then the transpose must be WRITTEN as a +/// composition of interleave primitives — and since the primitive itself +/// vectorizes from scalar source, no intrinsic override is earned, only a +/// method on the lane type. +#[inline(never)] +pub fn transpose_stage_u32x16(m: [U32x16; 16]) -> [U32x16; 16] { + let mut out = [U32x16::splat(0); 16]; + for i in 0..8 { + out[2 * i] = interleave_lo(m[2 * i], m[2 * i + 1]); + out[2 * i + 1] = interleave_hi(m[2 * i], m[2 * i + 1]); + } + out +} + +/// One butterfly exchange at block granularity `G` elements — the general +/// form of the whole unpack/permute family, parameterized by granularity +/// instead of one method per width. +/// +/// `G = 1` is the 32-bit unpack (`_mm256_unpack{lo,hi}_epi32`), `G = 2` the +/// 64-bit unpack (`_mm256_unpack{lo,hi}_epi64`), `G = 4` the 128-bit lane +/// exchange (`_mm256_permute2x128_si256`), and `G = 8` the 256-bit half +/// exchange a 512-bit-wide lane additionally needs. `G` is a const parameter, +/// so every shuffle pattern is compile-time constant — the same property a +/// hand-written intrinsic has, and a precondition for LLVM to select a +/// shuffle rather than an indexed copy. +#[inline(always)] +fn exchange(lo: U32x16, hi: U32x16) -> (U32x16, U32x16) { + let (l, h) = (lo.to_array(), hi.to_array()); + let mut nl = [0u32; 16]; + let mut nh = [0u32; 16]; + for c in 0..16 { + nl[c] = if c & G == 0 { l[c] } else { h[c ^ G] }; + nh[c] = if c & G != 0 { h[c] } else { l[c ^ G] }; + } + (U32x16::from_array(nl), U32x16::from_array(nh)) +} + +/// One full stage: pair every row `r` with `r | G` and exchange at +/// granularity `G`. +#[inline(always)] +fn stage(m: &mut [U32x16; 16]) { + for r in 0..16 { + if r & G == 0 { + let (a, b) = exchange::(m[r], m[r | G]); + m[r] = a; + m[r | G] = b; + } + } +} + +/// A COMPLETE, CORRECT 16x16 `u32` transpose composed from all four +/// granularities — which `transpose_stage_u32x16` above is not. +/// +/// This exists because that single-stage probe did not support the conclusion +/// drawn from it. It exercises only the 32-bit interleave shape, never the +/// 64-bit or 128/256-bit exchanges, and its helpers' whole-vector semantics +/// do not compose into a real transpose — so "the composed transpose stays +/// packed" was an inference, not a measurement. (Raised by codex on PR #265; +/// the finding was correct.) +/// +/// Algorithm: the standard recursive block transpose. For each granularity +/// `G` in 1, 2, 4, 8, pair row `r` with row `r | G` and swap the off-diagonal +/// blocks. Correctness is not asserted in prose — the driver checks this +/// against a naive nested-loop transpose on random input and aborts on +/// mismatch, so a packed-but-wrong result cannot be reported as a success. +#[inline(never)] +pub fn transpose_16x16_composed(m: [U32x16; 16]) -> [U32x16; 16] { + let mut m = m; + stage::<1>(&mut m); + stage::<2>(&mut m); + stage::<4>(&mut m); + stage::<8>(&mut m); + m +} + // ============================================================================ // Driver — runtime-derived inputs, every result consumed. // ============================================================================ @@ -375,5 +605,50 @@ fn main() { ); acc ^= ga.reduce_sum() ^ gb.reduce_sum() ^ gc.reduce_sum() ^ gd.reduce_sum(); + // ---- Group D ---- + let mut rand_u32x16 = || U32x16::from_array(std::array::from_fn(|_| rng.next() as u32)); + let (b3a, b3b, b3c, b3d) = blake3_g_u32x16( + black_box(rand_u32x16()), + black_box(rand_u32x16()), + black_box(rand_u32x16()), + black_box(rand_u32x16()), + black_box(rand_u32x16()), + black_box(rand_u32x16()), + ); + acc ^= (b3a.reduce_sum() ^ b3b.reduce_sum() ^ b3c.reduce_sum() ^ b3d.reduce_sum()) as u64; + + let inter = interleave_lo_u32x16(black_box(rand_u32x16()), black_box(rand_u32x16())); + acc ^= inter.reduce_sum() as u64; + + let mat: [U32x16; 16] = std::array::from_fn(|_| rand_u32x16()); + let transposed = transpose_16x16_u32(black_box(mat)); + acc ^= transposed + .iter() + .fold(0u64, |s, v| s ^ v.reduce_sum() as u64); + + let mat2: [U32x16; 16] = std::array::from_fn(|_| rand_u32x16()); + let staged = transpose_stage_u32x16(black_box(mat2)); + acc ^= staged.iter().fold(0u64, |s, v| s ^ v.reduce_sum() as u64); + + // The composed transpose is checked for CORRECTNESS against a naive + // nested-loop transpose before its codegen is reported. A shuffle network + // that is packed but wrong would otherwise read as a success, which is + // exactly the class of error this oracle exists to prevent. + let mat3: [U32x16; 16] = std::array::from_fn(|_| rand_u32x16()); + let composed = transpose_16x16_composed(black_box(mat3)); + { + let src: [[u32; 16]; 16] = std::array::from_fn(|i| mat3[i].to_array()); + let got: [[u32; 16]; 16] = std::array::from_fn(|i| composed[i].to_array()); + for i in 0..16 { + for j in 0..16 { + assert_eq!( + got[j][i], src[i][j], + "transpose_16x16_composed is not a transpose at ({i},{j})" + ); + } + } + } + acc ^= composed.iter().fold(0u64, |s, v| s ^ v.reduce_sum() as u64); + println!("simd-codegen-oracle: probes executed, combined checksum = {acc:#018x}"); } diff --git a/.claude/knowledge/simd-codegen-oracle/run.sh b/.claude/knowledge/simd-codegen-oracle/run.sh index 79c3caa8..e69b22bd 100755 --- a/.claude/knowledge/simd-codegen-oracle/run.sh +++ b/.claude/knowledge/simd-codegen-oracle/run.sh @@ -1,105 +1,179 @@ #!/bin/sh -# SIMD codegen oracle -- proves what actually vectorizes (crates/simd-codegen-oracle), -# in both directions: Group A probes must show packed AVX2 instructions, Group B -# probes must show none, Group C probes are reported without a pass/fail verdict -# (the open question this oracle was extended to answer). Mirrors the build/locate -# shape of scripts/neon-parity.sh / scripts/wasm-parity.sh, but the analysis itself -# (instruction classification, baseline comparison) lives in the Python helper -# scripts/codegen_oracle_analyze.py -- see its module docstring for the exact -# packed-vector / scalar-lane-arith / loop-control / memory / other classification -# rules and the documented honesty rule (a loop-counter decl is not lane -# arithmetic). +# SIMD codegen oracle -- measures what actually vectorizes, in both directions: +# Group A probes must show packed SIMD, Group B probes must show none, Group C +# probes are reported without a pass/fail verdict. # -# Usage: scripts/codegen-oracle.sh [target-triple] [-- --verbose] -# target-triple defaults to the host triple (`rustc -vV | grep ^host`). -# Everything after the target is forwarded to the analyzer (e.g. --verbose -# to print the raw instruction list per bucket). +# This is an ON-DEMAND instrument, not a CI job. Run it when a codegen question +# is genuinely open, record the answer in a doc, and stop. See README.md. +# +# Self-contained: builds a throwaway crate from probes.rs against the ndarray +# checkout this file lives in, so nothing needs to exist under crates/. +# +# Usage: sh run.sh [target-triple] [--verbose] +# target-triple defaults to the host triple. set -eu -ROOT="$(cd "$(dirname "$0")/.." && pwd)" -MANIFEST="$ROOT/crates/simd-codegen-oracle/Cargo.toml" +HERE="$(cd "$(dirname "$0")" && pwd)" +# .claude/knowledge/simd-codegen-oracle -> repo root is three levels up. +REPO="$(cd "$HERE/../../.." && pwd)" +if [ ! -f "$REPO/Cargo.toml" ]; then + echo "==> cannot locate the ndarray repo root from $HERE" >&2 + exit 91 +fi TARGET="${1:-}" case "$TARGET" in - "" | -*) - TARGET="$(rustc -vV | sed -n 's/^host: //p')" + "" | -*) TARGET="$(rustc -vV | sed -n 's/^host: //p')" ;; + *) shift ;; +esac + +# Baseline selection is EXACT-MATCH, never a family glob. A baseline records +# instruction counts measured on one triple at one microarchitecture level; +# another triple in the same family has different assembly syntax, a different +# ABI, and different symbol names, so comparing against it produces either a +# meaningless diff or a missing-symbol failure. An earlier version matched +# `x86_64-*`, which silently handed x86_64-pc-windows-msvc the Linux baseline. +# +# `baseline-x86_64-v3.toml` is named for the microarch level rather than the +# triple, so it gets one explicit exact-triple arm. Any other target needs its +# own `baseline-.toml`, exactly as README.md promises. +case "$TARGET" in + x86_64-unknown-linux-gnu) + BASELINE="$HERE/baseline-x86_64-v3.toml" + CPU="x86-64-v3" ;; *) - shift + BASELINE="$HERE/baseline-$TARGET.toml" + CPU="" ;; esac -# Remaining args (an optional `--` and/or analyzer flags like --verbose) forward as-is. -if [ "${1:-}" = "--" ]; then - shift -fi - -BASELINE="$ROOT/crates/simd-codegen-oracle/baselines/$TARGET.toml" if [ ! -f "$BASELINE" ]; then - echo "==> no baseline for target $TARGET at $BASELINE" >&2 + echo "==> no baseline for $TARGET at $BASELINE" >&2 + echo " A baseline is per-TRIPLE; there is deliberately no family fallback." >&2 + echo " (run with --verbose and record the observed counts to make one)" >&2 exit 90 fi -# The measured baseline is DECLARED here, never inherited from the ambient -# environment. This is load-bearing: +# `mktemp -d` and not "$TMPDIR/...-$$": a PID-derived name is predictable and +# `mkdir -p` happily accepts a path that already exists, so a local attacker +# could pre-create the directory (with symlinked children) and redirect the +# `cp` and heredoc writes below. mktemp creates atomically or fails. +SCRATCH="$(mktemp -d "${TMPDIR:-/tmp}/simd-codegen-oracle.XXXXXX")" || exit 91 +trap 'rm -rf "$SCRATCH"' EXIT +mkdir -p "$SCRATCH/src" +cp "$HERE/probes.rs" "$SCRATCH/src/main.rs" +cat > "$SCRATCH/Cargo.toml" < target-cpu=x86-64-v3 -# $ RUSTFLAGS="-D warnings" cargo build -v | grep target-cpu -> (empty) +# 1. A fresh scratch package has no lock, so cargo goes to the crates.io +# index before it will compile anything -- which fails outright in an +# offline or cold-cache environment even though the repo has a committed +# Cargo.lock sitting right there. +# 2. Resolving afresh means two oracle runs can measure two different +# dependency graphs. A tool whose whole job is reproducible measurement +# must not let its own inputs drift. # -# An oracle that inherits the ambient baseline therefore measures a -# DIFFERENT machine's codegen depending on where it runs, which is exactly -# the class of error it exists to catch. Passing `-C target-cpu` after the -# `--` puts it on the final rustc invocation, where it wins regardless of -# RUSTFLAGS. -case "$TARGET" in - x86_64-*) BASELINE_CPU="x86-64-v3" ;; - *) BASELINE_CPU="" ;; -esac +# The scratch package's dep closure is a subset of the repo's (one path dep on +# ndarray), so cargo amends the copied lock with the new root rather than +# re-resolving. `--offline` is attempted first and falls back to a normal +# build, so a cold cache still works. +if [ -f "$REPO/Cargo.lock" ]; then + cp "$REPO/Cargo.lock" "$SCRATCH/Cargo.lock" +fi -if [ -n "$BASELINE_CPU" ]; then - CPU_FLAG="-C target-cpu=$BASELINE_CPU" - echo "==> baseline: $TARGET @ target-cpu=$BASELINE_CPU (declared, not inherited)" +# The measured baseline is DECLARED, never inherited. Two independent reasons, +# and the first alone is NOT sufficient: +# +# 1. cargo resolves .cargo/config.toml from the CURRENT WORKING DIRECTORY, +# not from --manifest-path. Running from elsewhere silently misses the +# repo's config. Hence the `cd "$REPO"` below. +# 2. Even from the right directory, cargo's RUSTFLAGS env var REPLACES +# `[target.'cfg(...)'].rustflags` rather than merging with it. CI sets +# RUSTFLAGS="-D warnings" at workflow level, which drops the target-cpu +# pin entirely: +# $ cargo build -v | grep target-cpu -> x86-64-v3 +# $ RUSTFLAGS="-D warnings" cargo build -v | grep ... -> (nothing) +# +# Only passing `-C target-cpu` on the final rustc invocation survives both. +# An oracle that inherits its baseline measures a different machine depending +# on where it runs -- exactly the class of error it exists to catch. +if [ -n "$CPU" ]; then + CPU_FLAG="-C target-cpu=$CPU" + echo "==> baseline: $TARGET @ target-cpu=$CPU (declared, not inherited)" else CPU_FLAG="" echo "==> baseline: $TARGET @ target default (no target-cpu override)" fi -echo "==> building simd-codegen-oracle (--emit asm) for $TARGET" -if [ "$TARGET" = "$(rustc -vV | sed -n 's/^host: //p')" ]; then - # shellcheck disable=SC2086 - cargo rustc --release --manifest-path "$MANIFEST" -- --emit asm -C debuginfo=0 $CPU_FLAG -else - # shellcheck disable=SC2086 - cargo rustc --release --manifest-path "$MANIFEST" --target "$TARGET" -- --emit asm -C debuginfo=0 $CPU_FLAG -fi +cd "$REPO" +echo "==> building probes (--emit asm) for $TARGET" -# Excluded crate -> cargo places `deps/` either under the crate's own target -# dir or falls back to the workspace target dir, exactly like neon/wasm-parity. -ASM="" -for CAND_ROOT in "$ROOT/crates/simd-codegen-oracle/target" "$ROOT/target"; do +# Try the locked/offline build first (see the Cargo.lock copy above); fall back +# to a network build only if the local cache cannot satisfy the graph. +# --target-dir is PINNED to the scratch tree. Without it an ambient +# CARGO_TARGET_DIR / CARGO_BUILD_TARGET_DIR / build.target-dir would place the +# artifacts somewhere else entirely, so the `find "$SCRATCH/target"` below +# would come up empty and the scratch cleanup would leave real output behind. +build() { + # shellcheck disable=SC2086 if [ "$TARGET" = "$(rustc -vV | sed -n 's/^host: //p')" ]; then - CAND_DIR="$CAND_ROOT/release/deps" + cargo rustc "$@" --release --manifest-path "$SCRATCH/Cargo.toml" \ + --target-dir "$SCRATCH/target" -- \ + --emit asm -C debuginfo=0 $CPU_FLAG else - CAND_DIR="$CAND_ROOT/$TARGET/release/deps" + cargo rustc "$@" --release --manifest-path "$SCRATCH/Cargo.toml" \ + --target "$TARGET" --target-dir "$SCRATCH/target" -- \ + --emit asm -C debuginfo=0 $CPU_FLAG fi - if [ -d "$CAND_DIR" ]; then - FOUND="$(ls -t "$CAND_DIR"/simd_codegen_oracle-*.s 2>/dev/null | head -n1 || true)" - if [ -n "$FOUND" ]; then - ASM="$FOUND" - break - fi +} + +if [ -f "$SCRATCH/Cargo.lock" ] && build --offline 2>/dev/null; then + echo "==> built offline against the repository lockfile" + OFFLINE="--offline" +else + echo "==> offline build unavailable; resolving online" + OFFLINE="" + build +fi + +# EXECUTE the probes as well, on the host only. `--emit asm` replaces the +# default emit, so the build above links no binary and any runtime assertion +# inside a probe never runs. Some probes are self-checking -- notably +# `transpose_16x16_composed`, which verifies its shuffle network against a +# naive nested-loop transpose -- and a packed-but-WRONG network reported as a +# success would be precisely the error this tool exists to catch. +# +# Skipped when cross-compiling, where the host cannot run the artifact. +if [ "$TARGET" = "$(rustc -vV | sed -n 's/^host: //p')" ]; then + echo "==> executing probes (runtime self-checks)" + # shellcheck disable=SC2086 + if ! cargo run $OFFLINE --release --quiet --manifest-path "$SCRATCH/Cargo.toml" \ + --target-dir "$SCRATCH/target"; then + echo "==> a probe's runtime self-check FAILED -- codegen results are moot" >&2 + exit 93 fi -done +else + echo "==> cross-compiling; runtime self-checks skipped" +fi +ASM="$(find "$SCRATCH/target" -name 'simd_codegen_oracle-*.s' 2>/dev/null | head -1)" if [ -z "$ASM" ]; then - echo "==> could not locate emitted simd_codegen_oracle-*.s under $ROOT" >&2 - exit 91 + echo "==> no emitted assembly found under $SCRATCH/target" >&2 + exit 92 fi -echo "==> analyzing $ASM against $BASELINE" -python3 "$ROOT/scripts/codegen_oracle_analyze.py" "$ASM" "$BASELINE" "$@" +echo "==> analyzing $(basename "$ASM") against $(basename "$BASELINE")" +python3 "$HERE/analyze.py" "$ASM" "$BASELINE" "$@" diff --git a/.claude/knowledge/simd-one-spec-design.md b/.claude/knowledge/simd-one-spec-design.md index 35a41283..71df7260 100644 --- a/.claude/knowledge/simd-one-spec-design.md +++ b/.claude/knowledge/simd-one-spec-design.md @@ -173,6 +173,8 @@ shift-or composition written explicitly. - Ten currently-unlowered AVX2 int types: free. - The "is this fast enough?" argument: answered by a twenty-minute measurement during design, instead of debated. +- ~13k LoC of hand-maintained backend code: substantially reduced, with the + remainder being exactly the intrinsics that earn their place. ## On tooling — why this is NOT a CI job @@ -191,8 +193,11 @@ the answer goes in a doc. It is not re-answered on every commit. Run it when a codegen question is genuinely open: ```sh -see .claude/knowledge/simd-codegen-oracle/README.md # host, x86-64-v3 baseline -see .claude/knowledge/simd-codegen-oracle/README.md aarch64-unknown-linux-gnu +sh .claude/knowledge/simd-codegen-oracle/run.sh # host, x86-64-v3 baseline ``` -- ~13k LoC of hand-maintained backend code: substantially reduced, with the - remainder being exactly the intrinsics that earn their place. + +Cross-targets take a triple argument, but each needs its own +`baseline-.toml` first — only `baseline-x86_64-v3.toml` exists today, +so `run.sh aarch64-unknown-linux-gnu` currently exits 90 by design. Producing +that baseline is step 2 of the migration below, not a prerequisite anyone has +met yet.