docs: settle what actually compiles vendor/chacha20; board hygiene for #264/#265

.claude/board/AGENT_LOG.md

@@ -5,6 +5,40 @@
 > **Spawn protocol:** every agent reads this file before starting,
 > appends one entry on completion via `tee -a`.
 
+## 2026-07-29 — main thread — oracle repair + chacha20 reach + BLAKE3 gap
+
+**PRs:** #264 (merged), #265 (merged, `58521c3`)
+**Commits:** `87b9b4c`, `de4e4d1`, `3e72c5d`, `3c427ae`
+**Docs:** `.claude/knowledge/{blake3-on-ndarray-simd,chacha20-vendoring-blast-radius}.md`,
+`.claude/knowledge/simd-codegen-oracle/*`, `td-t22-asm-investigation.md`,
+`simd-one-spec-design.md`
+
+| item | outcome |
+|---|---|
+| oracle `run.sh` shipped unusable by #264's relocation | repaired, self-contained, 17/17 probes green from a clean master checkout |
+| `vendor/chacha20` blast radius | patch reaches 2 repos (ndarray, MedCare-rs), not 7; backend is compile-time gated on `avx512f` |
+| `AdaWorldAPI/stream-ciphers` | bare upstream mirror — same head sha, same `chacha20/` tree hash, 0 commits ahead |
+| BLAKE3 on `ndarray::simd` | needs one const-generic `exchange<G>` method, no intrinsics, no C |
+
+**Review:** 3 findings from codex, 6 from CodeRabbit, all addressed in
+`3c427ae`. The Major one (transpose probe did not validate a correct
+transpose) was a genuine methodological error, not a nit — see EPIPHANIES.
+
+**Corrected after merge, twice** (#266). Operator: *cargo is CI is github
+needs V3; dockerfile is V4*. Pass 1 of the correction concluded
+`Dockerfile.avx512` compiles and ships `ndarray_simd` — wrong; codex caught
+that both Dockerfiles run bare `cargo build --release` and `default-members`
+omits `crates/encryption`. Settled: no image compiles the backend; it is
+reached only by `-p encryption` / `--workspace` under an AVX-512 config, or
+wasm32+simd128. CI's missing global pin is separately confirmed deliberate
+(`ci.yaml:17-22`). EPIPHANIES entry names the two-axis pattern and the fact
+that the correction itself was the more confident error.
+
+**Open:** throughput bench vs `rust_avx2.rs` and vs upstream chacha20's own
+`avx512.rs`; the fork-vs-vendored-copy decision; whether the chacha20 cfg
+gate should gain a runtime arm.
+
+
 ## Fleet manifest
 
 | # | Agent | File | Model | Status |

.claude/board/EPIPHANIES.md

@@ -1,5 +1,132 @@
 # ndarray — Epiphanies (append-only)
 
+## 2026-07-29 — BLAKE3 needs a method surface, not intrinsics (measured)
+**Status:** FINDING
+**Scope:** @simd-savant domain:codec
+**Cross-ref:** PR #264, PR #265, `.claude/knowledge/blake3-on-ndarray-simd.md`,
+`.claude/knowledge/simd-codegen-oracle/`
+
+C is out of `blake3` (`features = ["pure"]`, #264 — 33 `.o` objects and
+`libblake3_avx512_assembly.a` gone). What remained was a second SIMD surface:
+2,910 lines of raw `core::arch` in its Rust backends.
+
+Census + four probes settle what a `ndarray::simd` backend would cost.
+`rust_avx2.rs` uses 15 distinct intrinsics; **twelve already exist on
+`U32x16`**. The other three families (`unpack{lo,hi}_epi32`,
+`unpack{lo,hi}_epi64`, `permute2x128` — 18 call sites) serve one purpose:
+`hash_many`'s transpose.
+
+| probe | packed | scalar on lane data |
+|---|---|---|
+| `blake3_g_u32x16` | 72 | 0 |
+| `interleave_lo_u32x16` | 11 | 0 |
+| `transpose_16x16_u32` (index loop) | **0** | 1 |
+| `transpose_16x16_composed` (4 stages, correctness-checked) | 79 | 0 |
+
+The compression core is free. The interleave *primitive* is free — a scalar
+index loop compiles to `vpermd` + `vpblendd`, a real two-source cross-lane
+permute. Only the **monolithic index-loop spelling** of the transpose fails;
+composed from `exchange<G>` at G = 1/2/4/8 it emits `vpunpcklqdq`, `vpermq`,
+`vpermpd`, `vinserti128` — the same family the intrinsic backend hand-writes.
+
+**So no intrinsic override is earned.** The gap is one const-generic method,
+~15 lines, no `unsafe` and no `core::arch`. That is the third consecutive
+time this oracle has turned "obviously needs intrinsics" into "already free"
+(after `saturating_abs`, `widening`, and `cross_lane_reverse`).
+
+**The methodological catch is the part worth keeping.** The first version of
+this finding rested on `transpose_stage_u32x16` — ONE 32-bit stage whose
+helpers, by their own doc comment, do not compose into a transpose. Codex and
+CodeRabbit independently flagged it. It was an inference wearing a
+measurement's clothes, published in the document that cites TD-T22 as the
+reason not to do exactly that. Two guards added in response:
+
+- the composed transpose is **checked against a naive reference and aborts on
+  mismatch**, control-tested by deleting `stage::<8>` to confirm it fires;
+- `run.sh` now **executes** the probe binary, because `--emit asm` links
+  nothing and the assertion had never actually run. A claim of verification
+  that does not verify is the same defect one level up.
+
+Still unmeasured, and stated as such: throughput vs `rust_avx2.rs`. "Emits
+packed shuffles" is not "is faster."
+
+
+## 2026-07-29 — A build claim has two axes; the correction erred on the second
+**Status:** FINDING
+**Scope:** @simd-savant @truth-architect domain:build-tiers
+**Cross-ref:** PR #265, `.claude/knowledge/chacha20-vendoring-blast-radius.md`,
+`.claude/knowledge/td-t22-asm-investigation.md`
+
+This repo has **three** build tiers, and an audit that reads only
+`.cargo/config.toml` sees one of them and concludes the opposite of the truth.
+
+| tier | target-cpu | purpose |
+|---|---|---|
+| CI (`ci.yaml`) | none globally; v4 in `tier4-avx512-check` | one binary, all ISAs, runtime `LazyLock<Tier>`; the `cross_test` matrix spans i686 and s390x, so a global pin is impossible |
+| `Dockerfile` | `x86-64-v3` | the portable image |
+| `Dockerfile.avx512` | `x86-64-v4` | the AVX-512 image |
+
+Operator's formulation: *cargo is CI is github needs V3; dockerfile is V4.*
+
+**A claim about "what a build does" has TWO axes, and I got each one wrong on
+a separate pass** — the second time while correcting the first:
+
+| axis | question | where the answer lives |
+|---|---|---|
+| `target-cpu` | which cfg-gated code is *selected*? | `.cargo/config*.toml`, `Dockerfile*` `RUSTFLAGS`, workflow `env:` incl. per-job |
+| package selection | is that crate *compiled at all*? | `default-members`, and the `-p` / `--workspace` flags on the actual command |
+
+Three passes on one paragraph:
+
+1. **First claim:** *"No default build runs `vendor/chacha20`'s
+   `ndarray_simd`."* Reasoned only from `.cargo/config.toml` pinning v3.
+   Right answer, incomplete reason.
+2. **First correction:** operator said *cargo is CI is github needs V3;
+   dockerfile is V4*, so I concluded `Dockerfile.avx512` compiles and ships
+   the backend. **Wrong.** I fixed the target-cpu axis and immediately erred
+   on the package-selection axis I still had not checked — both Dockerfiles
+   run bare `cargo build --release`, `default-members` omits
+   `crates/encryption`, and nothing else in that set pulls chacha20. Caught by
+   codex on #266.
+3. **Settled:** `ndarray_simd` is reached only by an explicit
+   `-p encryption` / `--workspace` build under an AVX-512 config, or by
+   wasm32+`simd128`. No image compiles it.
+
+A separate instance of the same shape, same week: *"CI has been testing
+different machine code than anyone reviews"*, filed as a ⚠ defect. It is the
+design, and `ci.yaml:17-22` says so in prose I had not read — a global pin
+collides with the non-x86 cross_test matrix and contradicts the
+runtime-dispatch intent.
+
+**The pattern to name:** every individual measurement was correct. The error
+was always the *scope quantifier* — "no build", "CI", "ships" — attached to
+evidence drawn from one file. `rustc --print cfg` told me what v3 lacks; it
+could not tell me which tiers exist. Knowing the tiers still could not tell me
+what each tier builds.
+
+**And the correction is as dangerous as the original claim.** Being handed the
+missing piece feels like completion, so pass 2 shipped faster and with more
+confidence than pass 1, and was more wrong. A correction is a new claim and
+earns no discount on verification.
+
+Consequence, concretely — before any "no build does X" / "X ships with Y":
+
+```console
+find . -iname 'Dockerfile*'                 # which tiers exist
+grep -n 'RUN cargo' Dockerfile*             # what each one actually builds
+sed -n '/default-members/,/]/p' Cargo.toml  # what a bare build selects
+cargo tree -p <pkg> -i <dep>                # per package, not per workspace
+```
+
+A second-order note worth keeping: ndarray's own SIMD upgrades itself at run
+time via `LazyLock<Tier>` even in a v3 build, but `vendor/chacha20`'s gate is
+`#[cfg(target_feature = "avx512f")]` — compile-time, no runtime fallback. So a
+v3 image on AVX-512 silicon has ndarray's kernels upgrading while the chacha20
+keystream stays on RustCrypto's backends. Compile-time and runtime dispatch
+living in one binary is not a contradiction, but it means "this build is v3"
+does not settle what any given subsystem selects.
+
+
 ## 2026-04-19 — Prompt↔PR ledger is 10⁷× cheaper than code grep
 **Status:** FINDING
 **Scope:** @workspace-primer domain:bookkeeping

.claude/knowledge/chacha20-vendoring-blast-radius.md

@@ -73,38 +73,124 @@ $ rustc --print cfg -Ctarget-cpu=x86-64-v4 | grep avx512f
 target_feature="avx512f"
 ```
 
-| build | target-cpu | `avx512f` | backend compiled |
+### The tiers, and what each one actually builds
+
+**Two independent axes decide this, and reading either one alone gets it
+wrong.** Axis one is `target-cpu`, which decides *which backend* is selected.
+Axis two is **package selection**, which decides whether `chacha20` is
+compiled at all.
+
+| tier | target-cpu | builds `encryption`? | chacha20 backend |
 |---|---|---|---|
-| ndarray, default | `x86-64-v3` (`.cargo/config.toml`) | absent | RustCrypto soft/avx2/sse2 |
-| ndarray, `--config .cargo/config-avx512.toml` | `x86-64-v4` | present | **`ndarray_simd`** |
-| MedCare-rs, default | none — no `.cargo/config.toml` at all | absent | RustCrypto soft/avx2/sse2 |
-| wasm32 + `simd128` | — | n/a | **`ndarray_simd`** |
-| the five unpatched repos | — | — | not present in the source they link |
+| **CI** (`ci.yaml`) | none globally | no | not compiled |
+| CI `tier4-avx512-check` | v4, per-job | no — `-p ndarray` only | not compiled |
+| **`Dockerfile`** | v3 | **no** — bare `cargo build` | **not compiled** |
+| **`Dockerfile.avx512`** | v4 | **no** — bare `cargo build` | **not compiled** |
+| dev `cargo build -p encryption` | v3 (`.cargo/config.toml`) | yes | RustCrypto avx2/sse2 |
+| …with `--config .cargo/config-avx512.toml` | sapphirerapids | yes | **`ndarray_simd`** |
+| wasm32 + `simd128` | — | if selected | **`ndarray_simd`** |
+
+The Dockerfile rows are the ones that surprise. Both images run bare
+`cargo build --release`, which selects `default-members`
+(`Cargo.toml:436-443`): `.`, `ndarray-rand`, `crates/ndarray-gen`,
+`crates/numeric-tests`, `crates/serialization-tests`. **`crates/encryption`
+is not among them**, and nothing else in that set depends on chacha20 —
+verified per-package:
 
-**No default build of any repo in the workspace runs `ndarray_simd`.** Two
-opt-in configurations do: an explicitly-selected AVX-512 x86_64 build, and a
-wasm32 build with `simd128`.
+```console
+$ for p in ndarray ndarray-rand ndarray-gen numeric-tests serialization-tests; do
+      cargo tree -p "$p" -i chacha20; done
+error: package ID specification `chacha20` did not match any packages   # x5
+
+$ cargo tree -p encryption -i chacha20
+chacha20 v0.9.1 (/workspace/ndarray/vendor/chacha20)
+└── chacha20poly1305 v0.10.1
+    └── encryption v0.1.0 (/workspace/ndarray/crates/encryption)
+```
 
-This is not a defect — the fallthrough is correct and the comments in
-MedCare-rs's manifest describe it accurately ("non-avx512 builds fall through
-to RustCrypto's own backends"). It is a scope fact that keeps being stated
-one size too large.
+`encryption` *is* a workspace member (via `crates/*`), so `--workspace` or
+`-p encryption` reaches it — neither Dockerfile passes either.
+
+**So `ndarray_simd` is reached only by an explicit `-p encryption` /
+`--workspace` build under an AVX-512 config, or by a wasm32+`simd128`
+build.** No image in this repo compiles it.
+
+### Correction history — three passes on one paragraph
+
+Worth keeping visible, because the failure mode repeated:
+
+1. **First claim:** "no default build runs `ndarray_simd`", reasoned purely
+   from `.cargo/config.toml` pinning v3. Right answer, incomplete reason.
+2. **First correction:** the operator pointed out *cargo is CI is github
+   needs V3; dockerfile is V4*, so I concluded `Dockerfile.avx512` compiles
+   and ships the backend. **Wrong** — I fixed the `target-cpu` axis and
+   introduced a new error on the package-selection axis I still had not
+   checked. (Caught by codex on PR #266.)
+3. **This version:** both axes checked with `cargo tree`, per package.
+
+The operator's tier statement was correct throughout; what it does not imply
+is that either image builds the crate that pulls chacha20.
+
+Two things none of this changes:
+
+- The **patch-reach** table above stands. It is about which repos link the
+  vendored source when they *do* build it.
+- The gate is **compile-time**, not runtime. `Dockerfile`'s own comment notes
+  that ndarray's `simd.rs` detects AVX-512 at run time via `LazyLock<Tier>`
+  even in a v3 build — but `vendor/chacha20/src/backends.rs` keys on
+  `#[cfg(target_feature = "avx512f")]`, resolved by the compiler with no such
+  fallback. So even in a hypothetical v4 image that *did* build `encryption`,
+  the two subsystems would dispatch by different mechanisms.
+
+### The v4 arm is covered in CI, per-job
+
+CI's global env carries no `target-cpu`, and `ci.yaml:17-22` records why:
+it collides with the `cross_test` matrix (`i686` is 32-bit, `s390x` is not
+x86 at all) and contradicts the one-binary + runtime-dispatch design intent.
+Jobs needing a higher tier opt in individually — `tier4-avx512-check` sets
+`CARGO_TARGET_X86_64_UNKNOWN_LINUX_GNU_RUSTFLAGS: "-D warnings
+-Ctarget-cpu=x86-64-v4"`, deliberately the per-target form rather than plain
+`RUSTFLAGS`, because `RUSTFLAGS` also applies to host build scripts and those
+SIGILL on a runner without AVX-512 silicon.
+
+**That job covers ndarray's v4 arm and nothing else.** Both its steps are
+`cargo check --target=x86_64-unknown-linux-gnu -p ndarray --features
+approx,serde,rayon` (the second adding `hpc-extras`) — package-scoped to
+`ndarray`, so `crates/encryption` and therefore `vendor/chacha20` are outside
+it. **No CI job compiles the chacha20 AVX-512 backend**, and none of the
+coverage claimed here extends to it. Closing that gap would take an explicit
+step such as:
 
-## One manifest comment overstates it
+```console
+$ CARGO_TARGET_X86_64_UNKNOWN_LINUX_GNU_RUSTFLAGS="-Ctarget-cpu=x86-64-v4" \
+      cargo check --target=x86_64-unknown-linux-gnu -p encryption
+```
+
+Not added here — adding a CI job is a product change, and this is a docs PR.
+
+## The manifest comment, re-read
 
 `Cargo.toml:481-482` reads:
 
 > This transitively accelerates the `encryption` crate's XChaCha20-Poly1305
 > keystream … **on any x86_64+avx512f build (the workspace's
 > `target-cpu=x86-64-v4`)**.
 
-The workspace's pinned target-cpu is **v3**, in `.cargo/config.toml`. `v4`
-is the opt-in `.cargo/config-avx512.toml`. The conditional half of the
-sentence is right; the parenthetical asserts the default build satisfies it,
-and it does not.
+An earlier version of this audit filed that parenthetical as a factual error,
+on the grounds that the workspace pins v3; a later version withdrew the
+filing, on the grounds that `Dockerfile.avx512` is a v4 build of this
+workspace. **Both were partly wrong.**
+
+Settled: "the workspace's `target-cpu=x86-64-v4`" does name a real tier, so
+the parenthetical is imprecise rather than false — it is not the tier
+`cargo build` selects. But the sentence's *main* clause is the weaker part.
+"This transitively accelerates the `encryption` crate's … keystream" holds
+only for a build that actually selects `encryption`, and **no image in this
+repo does**. The acceleration is real and reachable; it is not automatic, and
+nothing ships with it today.
 
-**Not edited here.** Manifest string changes in this repo need an operator
-ruling; the finding is recorded instead of applied.
+Still **not edited**, for the same reason as before: manifest string changes
+need a ruling. Recorded here so the next reader is not misled by it.
 
 ## The fork the P0 rule names is not this tree
 
@@ -161,21 +247,27 @@ Three consequences worth stating plainly:
 
 ## What is NOT claimed here
 
-- Not that the vendored backend is wrong. It is untested by default builds,
-  which is a different statement.
-- Not that the patch should be removed. That is a decision, and it depends on
-  (3) above plus the AVX-512 deployment question, neither of which this audit
-  settles.
+- Not that the vendored backend is wrong, or unused. `Dockerfile.avx512`
+  compiles it; it is the shipped path on v4 silicon.
+- Not that the patch should be removed. That depends on (3) above, which is a
+  measurement nobody has taken.
 - Not that MedCare-rs's wiring is incorrect. It is the one consumer that
   declared its patch deliberately and documented the fallthrough honestly.
 
-## Open, for an operator ruling
+## Still open
 
-1. **Point `vendor/chacha20` at the fork, or keep the copy?** The P0 rule
-   says depend on the AdaWorldAPI fork. Today the tree is neither — a copy of
+1. **Point `vendor/chacha20` at the fork, or keep the copy?** The P0 rule says
+   depend on the AdaWorldAPI fork. Today the tree is neither — a hand copy of
    a version the fork does not carry, with `repository` rewritten to point at
-   ndarray itself.
-2. **Does `ndarray_simd` still beat upstream's own `avx512.rs`?** Answerable
-   with the oracle. Unmeasured today, on both sides.
-3. **The v3/v4 manifest comment** — correct in place, or leave the record and
-   annotate?
+   ndarray itself. The fork is provably a bare upstream mirror (same head sha,
+   same `chacha20/` tree hash), so "track the fork" currently means "track
+   upstream 0.10.1", which crosses a `cipher` major.
+2. **Does `ndarray_simd` still beat upstream's own `avx512.rs`?** This is the
+   load-bearing question now that upstream ships an AVX-512 backend of its
+   own, and it is the one that decides (1). Unmeasured on both sides. The
+   oracle answers the codegen half; the throughput half needs a bench.
+3. **Should `vendor/chacha20`'s cfg gate get a runtime arm?** Today it is
+   compile-time only, so a v3 image on AVX-512 silicon runs RustCrypto's
+   backends for the keystream while ndarray's own kernels upgrade via
+   `LazyLock<Tier>`. Whether that asymmetry is worth closing is a design call,
+   not a bug report — and it only matters if (2) says the ndarray lane wins.