|
| 1 | +# TD-T22 — the AVX2 int-polyfill investigation (CLOSED, no gap) |
| 2 | + |
| 3 | +## READ BY: |
| 4 | +- Anyone about to "lower the AVX2 scalar polyfills to native `__m256i`" |
| 5 | +- `simd-savant`, `truth-architect` — before accepting a SIMD-lowering proposal |
| 6 | +- Any session reading a `🟠 scalar polyfill` cell in the parity matrix |
| 7 | + |
| 8 | +## P0 TRIGGER |
| 9 | +About to hand-write `__m256i` wrappers for `U32x8` / `U32x16` / `U16x32` / |
| 10 | +`U64x4` / `I32x8` / `I64x4` because the parity matrix says "scalar polyfill"? |
| 11 | +**Read this first. The matrix marks a SOURCE-level property, not a codegen |
| 12 | +one, and the codegen gap does not exist.** |
| 13 | + |
| 14 | +--- |
| 15 | + |
| 16 | +## The question TD-T22 actually asked |
| 17 | + |
| 18 | +`td-simd-tier-audit.md:339` — `TD-T22 | – | – | Investigation only — needs |
| 19 | +simd_avx2.rs read first`. `CHACHA20_MATRYOSHKA_PLAN.md:104` — *"avx2 native |
| 20 | +`U32x16` — the TD-SIMD-3 lowering; **still optional**, the scalar polyfill is |
| 21 | +correct meanwhile."* |
| 22 | + |
| 23 | +The ticket asked whether the `avx2_int_type!` scalar-storage polyfills |
| 24 | +actually cost anything. It did not ask for a lowering. This note answers it. |
| 25 | + |
| 26 | +## Answer |
| 27 | + |
| 28 | +**No gap. The "scalar polyfill" is not scalar in the emitted binary.** |
| 29 | +`.cargo/config.toml` pins `-Ctarget-cpu=x86-64-v3` (AVX2+FMA) for **every** |
| 30 | +x86_64 build including CI, so LLVM auto-vectorizes the macro's |
| 31 | +`for i in 0..N { … }` bodies into packed AVX2. Measured below: **zero scalar |
| 32 | +ALU instructions**, and the ARX loop hits the theoretical instruction minimum. |
| 33 | + |
| 34 | +## Method |
| 35 | + |
| 36 | +Branch at `1ff8171`. `U32x16` is `avx2_int_type!(U32x16, u32, 16, 0u32)` |
| 37 | +(`simd_avx2.rs:1542`) — macro-generated `[u32; 16]` storage, confirmed: no |
| 38 | +hand-written `pub struct U32x16` exists in that file. |
| 39 | + |
| 40 | +```rust |
| 41 | +// examples/td_t22_probe.rs — #[inline(never)] so each survives as a symbol |
| 42 | +use ndarray::simd::U32x16; |
| 43 | + |
| 44 | +#[inline(never)] |
| 45 | +pub fn arx_quarter_round(a: U32x16, b: U32x16) -> U32x16 { |
| 46 | + let s = a + b; |
| 47 | + let x = s ^ b; |
| 48 | + x.rotate_left(16) |
| 49 | +} |
| 50 | + |
| 51 | +#[inline(never)] |
| 52 | +pub fn arx_ten_rounds(mut st: [U32x16; 4]) -> [U32x16; 4] { |
| 53 | + for _ in 0..10 { |
| 54 | + st[0] = st[0] + st[1]; |
| 55 | + st[3] = (st[3] ^ st[0]).rotate_left(16); |
| 56 | + st[2] = st[2] + st[3]; |
| 57 | + st[1] = (st[1] ^ st[2]).rotate_left(12); |
| 58 | + st[0] = st[0] + st[1]; |
| 59 | + st[3] = (st[3] ^ st[0]).rotate_left(8); |
| 60 | + st[2] = st[2] + st[3]; |
| 61 | + st[1] = (st[1] ^ st[2]).rotate_left(7); |
| 62 | + } |
| 63 | + st |
| 64 | +} |
| 65 | + |
| 66 | +#[inline(never)] |
| 67 | +pub fn red_sum(a: U32x16) -> u32 { a.reduce_sum() } |
| 68 | +``` |
| 69 | + |
| 70 | +```sh |
| 71 | +cargo rustc --release --example td_t22_probe -p ndarray -- --emit asm -C debuginfo=0 |
| 72 | +# then histogram the instructions between each symbol's .type/@function label |
| 73 | +# and its retq, in target/release/examples/td_t22_probe-*.s |
| 74 | +``` |
| 75 | + |
| 76 | +## Measured output |
| 77 | + |
| 78 | +**`arx_quarter_round`** — `(a+b)^b`, then `rotate_left(16)`: |
| 79 | + |
| 80 | +| instr | count | |
| 81 | +|---|---| |
| 82 | +| `vpaddd` | 2 | |
| 83 | +| `vpxor` | 2 | |
| 84 | +| `vpshufb` | 2 | |
| 85 | +| `vmovdqa` | 5 | |
| 86 | +| **scalar arithmetic on lane data** | **0** | |
| 87 | + |
| 88 | +16 u32 lanes = 2 ymm registers, so 2 packed ops per lane-op is exactly one |
| 89 | +instruction per half. LLVM strength-reduced `rotate_left(16)` into a **byte |
| 90 | +shuffle** (`vpshufb`) — cheaper than the `shl|shr|or` triple a hand-written |
| 91 | +intrinsic version would emit. |
| 92 | + |
| 93 | +**`arx_ten_rounds`** — the full ChaCha20 double-round over `[U32x16; 4]`: |
| 94 | + |
| 95 | +| instr | count | |
| 96 | +|---|---| |
| 97 | +| `vpaddd` | 8 | |
| 98 | +| `vpxor` | 8 | |
| 99 | +| `vpshufb` | 6 | |
| 100 | +| `vpsrld` / `vpslld` / `vpor` | 4 / 4 / 4 | |
| 101 | +| `jne` | 1 | |
| 102 | +| **scalar arithmetic on lane data** | **0** | |
| 103 | + |
| 104 | +The `jne` shows this is one rolled loop body, so the counts are per |
| 105 | +iteration. **This is the instruction-count floor:** 4 `U32x16` adds = 64 u32 |
| 106 | +lanes; on 256-bit AVX2 that is 8 ymm adds minimum, and exactly 8 `vpaddd` |
| 107 | +were emitted. Rotates by 16 and 8 fold to `vpshufb` (byte-granular); rotates |
| 108 | +by 12 and 7 use the `vpsrld`/`vpslld`/`vpor` triple. **There is no headroom |
| 109 | +for a hand-written version to recover.** |
| 110 | + |
| 111 | +**`red_sum`** — `reduce_sum`: |
| 112 | + |
| 113 | +| instr | count | |
| 114 | +|---|---| |
| 115 | +| `vpaddd` | 4 | |
| 116 | +| `vpshufd` | 2 | |
| 117 | +| `vextracti128` | 1 | |
| 118 | +| `vmovd` | 1 | |
| 119 | +| **scalar arithmetic on lane data** | **0** | |
| 120 | + |
| 121 | +A textbook logarithmic horizontal-reduction tree, not the scalar |
| 122 | +`wrapping_add` fold the source literally spells out. |
| 123 | + |
| 124 | +**Precision on "0 scalar arithmetic on lane data".** A broad sweep of ALL |
| 125 | +non-vector instructions across the three symbols returns exactly: |
| 126 | +`3 retq`, `1 movl`, `1 jne`, `1 decl`. The `movl`/`decl`/`jne` are the |
| 127 | +`arx_ten_rounds` loop counter and branch — loop control, not lane |
| 128 | +arithmetic. So the honest claim is **no scalar op touches lane data**, not |
| 129 | +"no scalar instruction exists": one `decl` does, and it decrements the trip |
| 130 | +count. Every `u32` lane operation in all three probes is a packed AVX2 |
| 131 | +instruction. |
| 132 | + |
| 133 | +## The same result holds for the float side |
| 134 | + |
| 135 | +`F32x16::mul_add` (`simd_avx2.rs`) is likewise written as |
| 136 | +`to_array()` → scalar loop → `from_array()`. `add_mul_f32` built on it emits |
| 137 | +**`vfmadd213ps`** — real fused multiply-add, one rounding, mantissa |
| 138 | +preserved. The doc comment at `simd_ops.rs:132` claiming "AVX2 + FMA: |
| 139 | +`_mm256_fmadd_ps`" is correct about the *emitted code* despite the scalar |
| 140 | +source. `array_chunks` / `array_windows` (+ `_checked`) already exist in |
| 141 | +`simd_ops.rs` as the slice-level primitives. |
| 142 | + |
| 143 | +## Consequences |
| 144 | + |
| 145 | +1. **TD-T22 is CLOSED as "no gap".** The `🟠 scalar polyfill` cells in the |
| 146 | + parity matrix and the `⏳` in `agnostic-surface-cpu-matrix.md` mark a |
| 147 | + **source-level** property. They are accurate as written and must not be |
| 148 | + read as a performance defect. |
| 149 | +2. **Do not hand-lower these types for codegen reasons.** A lowering can |
| 150 | + only be justified by properties LLVM does *not* give you: |
| 151 | + - `#[repr(align(64))]` cacheline guarantee (the polyfill HAS it; a |
| 152 | + `#[repr(transparent)]` `__m256i` wrapper LOSES it — measured: |
| 153 | + `U32x8` size 64→32, `U32x16` align 64→32), |
| 154 | + - ABI shape across a non-inlined boundary, |
| 155 | + - independence from `opt-level` (auto-vectorization does not hold at |
| 156 | + `-O0`/`-O1`) and from LLVM version drift. |
| 157 | + Those are real but small, and none of them is a speed argument. |
| 158 | +3. **`U32x8` must not become `U32x16`'s building block.** Operator ruling, |
| 159 | + 2026-07-28: a half-width type composing the lane the substrate actually |
| 160 | + uses is an absolute no-go — it splits the lane vocabulary for no gain. |
| 161 | +4. **The guard that was missing is a codegen oracle, not a parity harness.** |
| 162 | + x86_64 is the CI host, so `cargo test` already runs the AVX2 tier |
| 163 | + natively and `simd.rs`'s `u32x16_arx_ops_match_scalar` already gates every |
| 164 | + backend's ARX triple against scalar. What did not exist — and what would |
| 165 | + have answered this ticket in twenty minutes — is an **asm-diff / bench** |
| 166 | + check. Adding one to CI is the sanctioned follow-up. |
| 167 | + |
| 168 | +## Provenance |
| 169 | + |
| 170 | +Investigated 2026-07-28 after a lowering PR (closed unmerged, ndarray #261) |
| 171 | +shipped ~700 lines of hand-written intrinsics on the premise that these |
| 172 | +polyfills executed scalar code. They do not. The PR was closed on the |
| 173 | +operator's `U32x8`-composition ruling; this investigation is the deliverable |
| 174 | +the ticket originally asked for. |
0 commit comments