TD-T22: close as investigated, no gap — the AVX2 int polyfills are not scalar #262
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,174 @@ | ||
| # TD-T22 — the AVX2 int-polyfill investigation (CLOSED, no gap) | ||
|
|
||
| ## READ BY: | ||
| - Anyone about to "lower the AVX2 scalar polyfills to native `__m256i`" | ||
| - `simd-savant`, `truth-architect` — before accepting a SIMD-lowering proposal | ||
| - Any session reading a `🟠 scalar polyfill` cell in the parity matrix | ||
|
|
||
| ## P0 TRIGGER | ||
| About to hand-write `__m256i` wrappers for `U32x8` / `U32x16` / `U16x32` / | ||
| `U64x4` / `I32x8` / `I64x4` because the parity matrix says "scalar polyfill"? | ||
| **Read this first. The matrix marks a SOURCE-level property, not a codegen | ||
| one, and the codegen gap does not exist.** | ||
|
|
||
| --- | ||
|
|
||
| ## The question TD-T22 actually asked | ||
|
|
||
| `td-simd-tier-audit.md:339` — `TD-T22 | – | – | Investigation only — needs | ||
| simd_avx2.rs read first`. `CHACHA20_MATRYOSHKA_PLAN.md:104` — *"avx2 native | ||
| `U32x16` — the TD-SIMD-3 lowering; **still optional**, the scalar polyfill is | ||
| correct meanwhile."* | ||
|
|
||
| The ticket asked whether the `avx2_int_type!` scalar-storage polyfills | ||
| actually cost anything. It did not ask for a lowering. This note answers it. | ||
|
|
||
| ## Answer | ||
|
|
||
| **No gap. The "scalar polyfill" is not scalar in the emitted binary.** | ||
| `.cargo/config.toml` pins `-Ctarget-cpu=x86-64-v3` (AVX2+FMA) for **every** | ||
| x86_64 build including CI, so LLVM auto-vectorizes the macro's | ||
| `for i in 0..N { … }` bodies into packed AVX2. Measured below: **zero scalar | ||
| ALU instructions**, and the ARX loop hits the theoretical instruction minimum. | ||
|
Comment on lines
+28
to
+32
There was a problem hiding this comment.
The Useful? React with 👍 / 👎.
Comment on lines
+26
to
+32
Contributor
There was a problem hiding this comment. 🎯 Functional Correctness | 🟠 Major | 🏗️ Heavy lift Align the closure claims with the measured probe scope. The supplied evidence covers
📍 Affects 3 files
🤖 Prompt for AI Agents |
||
|
|
||
| ## Method | ||
|
|
||
| Branch at `1ff8171`. `U32x16` is `avx2_int_type!(U32x16, u32, 16, 0u32)` | ||
| (`simd_avx2.rs:1542`) — macro-generated `[u32; 16]` storage, confirmed: no | ||
| hand-written `pub struct U32x16` exists in that file. | ||
|
|
||
| ```rust | ||
| // examples/td_t22_probe.rs — #[inline(never)] so each survives as a symbol | ||
| use ndarray::simd::U32x16; | ||
|
|
||
| #[inline(never)] | ||
| pub fn arx_quarter_round(a: U32x16, b: U32x16) -> U32x16 { | ||
| let s = a + b; | ||
| let x = s ^ b; | ||
| x.rotate_left(16) | ||
| } | ||
|
|
||
| #[inline(never)] | ||
| pub fn arx_ten_rounds(mut st: [U32x16; 4]) -> [U32x16; 4] { | ||
| for _ in 0..10 { | ||
| st[0] = st[0] + st[1]; | ||
| st[3] = (st[3] ^ st[0]).rotate_left(16); | ||
| st[2] = st[2] + st[3]; | ||
| st[1] = (st[1] ^ st[2]).rotate_left(12); | ||
| st[0] = st[0] + st[1]; | ||
| st[3] = (st[3] ^ st[0]).rotate_left(8); | ||
| st[2] = st[2] + st[3]; | ||
| st[1] = (st[1] ^ st[2]).rotate_left(7); | ||
| } | ||
| st | ||
| } | ||
|
|
||
| #[inline(never)] | ||
| pub fn red_sum(a: U32x16) -> u32 { a.reduce_sum() } | ||
| ``` | ||
|
|
||
| ```sh | ||
| cargo rustc --release --example td_t22_probe -p ndarray -- --emit asm -C debuginfo=0 | ||
| # then histogram the instructions between each symbol's .type/@function label | ||
| # and its retq, in target/release/examples/td_t22_probe-*.s | ||
| ``` | ||
|
|
||
| ## Measured output | ||
|
|
||
| **`arx_quarter_round`** — `(a+b)^b`, then `rotate_left(16)`: | ||
|
|
||
| | instr | count | | ||
| |---|---| | ||
| | `vpaddd` | 2 | | ||
| | `vpxor` | 2 | | ||
| | `vpshufb` | 2 | | ||
| | `vmovdqa` | 5 | | ||
| | **scalar arithmetic on lane data** | **0** | | ||
|
|
||
| 16 u32 lanes = 2 ymm registers, so 2 packed ops per lane-op is exactly one | ||
| instruction per half. LLVM strength-reduced `rotate_left(16)` into a **byte | ||
| shuffle** (`vpshufb`) — cheaper than the `shl|shr|or` triple a hand-written | ||
| intrinsic version would emit. | ||
|
|
||
| **`arx_ten_rounds`** — the full ChaCha20 double-round over `[U32x16; 4]`: | ||
|
There was a problem hiding this comment.
This label is inaccurate: Useful? React with 👍 / 👎. |
||
|
|
||
| | instr | count | | ||
| |---|---| | ||
| | `vpaddd` | 8 | | ||
| | `vpxor` | 8 | | ||
| | `vpshufb` | 6 | | ||
| | `vpsrld` / `vpslld` / `vpor` | 4 / 4 / 4 | | ||
| | `jne` | 1 | | ||
| | **scalar arithmetic on lane data** | **0** | | ||
|
|
||
| The `jne` shows this is one rolled loop body, so the counts are per | ||
| iteration. **This is the instruction-count floor:** 4 `U32x16` adds = 64 u32 | ||
| lanes; on 256-bit AVX2 that is 8 ymm adds minimum, and exactly 8 `vpaddd` | ||
| were emitted. Rotates by 16 and 8 fold to `vpshufb` (byte-granular); rotates | ||
| by 12 and 7 use the `vpsrld`/`vpslld`/`vpor` triple. **There is no headroom | ||
| for a hand-written version to recover.** | ||
|
coderabbitai[bot] marked this conversation as resolved.
|
||
|
|
||
| **`red_sum`** — `reduce_sum`: | ||
|
|
||
| | instr | count | | ||
| |---|---| | ||
| | `vpaddd` | 4 | | ||
| | `vpshufd` | 2 | | ||
| | `vextracti128` | 1 | | ||
| | `vmovd` | 1 | | ||
| | **scalar arithmetic on lane data** | **0** | | ||
|
|
||
| A textbook logarithmic horizontal-reduction tree, not the scalar | ||
| `wrapping_add` fold the source literally spells out. | ||
|
|
||
| **Precision on "0 scalar arithmetic on lane data".** A broad sweep of ALL | ||
| non-vector instructions across the three symbols returns exactly: | ||
| `3 retq`, `1 movl`, `1 jne`, `1 decl`. The `movl`/`decl`/`jne` are the | ||
| `arx_ten_rounds` loop counter and branch — loop control, not lane | ||
| arithmetic. So the honest claim is **no scalar op touches lane data**, not | ||
| "no scalar instruction exists": one `decl` does, and it decrements the trip | ||
| count. Every `u32` lane operation in all three probes is a packed AVX2 | ||
| instruction. | ||
|
|
||
| ## The same result holds for the float side | ||
|
|
||
| `F32x16::mul_add` (`simd_avx2.rs`) is likewise written as | ||
| `to_array()` → scalar loop → `from_array()`. `add_mul_f32` built on it emits | ||
| **`vfmadd213ps`** — real fused multiply-add, one rounding, mantissa | ||
| preserved. The doc comment at `simd_ops.rs:132` claiming "AVX2 + FMA: | ||
| `_mm256_fmadd_ps`" is correct about the *emitted code* despite the scalar | ||
| source. `array_chunks` / `array_windows` (+ `_checked`) already exist in | ||
| `simd_ops.rs` as the slice-level primitives. | ||
|
|
||
| ## Consequences | ||
|
|
||
| 1. **TD-T22 is CLOSED as "no gap".** The `🟠 scalar polyfill` cells in the | ||
| parity matrix and the `⏳` in `agnostic-surface-cpu-matrix.md` mark a | ||
| **source-level** property. They are accurate as written and must not be | ||
| read as a performance defect. | ||
| 2. **Do not hand-lower these types for codegen reasons.** A lowering can | ||
| only be justified by properties LLVM does *not* give you: | ||
| - `#[repr(align(64))]` cacheline guarantee (the polyfill HAS it; a | ||
| `#[repr(transparent)]` `__m256i` wrapper LOSES it — measured: | ||
| `U32x8` size 64→32, `U32x16` align 64→32), | ||
| - ABI shape across a non-inlined boundary, | ||
| - independence from `opt-level` (auto-vectorization does not hold at | ||
| `-O0`/`-O1`) and from LLVM version drift. | ||
| Those are real but small, and none of them is a speed argument. | ||
| 3. **`U32x8` must not become `U32x16`'s building block.** Operator ruling, | ||
| 2026-07-28: a half-width type composing the lane the substrate actually | ||
| uses is an absolute no-go — it splits the lane vocabulary for no gain. | ||
| 4. **The guard that was missing is a codegen oracle, not a parity harness.** | ||
| x86_64 is the CI host, so `cargo test` already runs the AVX2 tier | ||
| natively and `simd.rs`'s `u32x16_arx_ops_match_scalar` already gates every | ||
| backend's ARX triple against scalar. What did not exist — and what would | ||
| have answered this ticket in twenty minutes — is an **asm-diff / bench** | ||
| check. Adding one to CI is the sanctioned follow-up. | ||
|
Comment on lines
+152
to
+157
Contributor
There was a problem hiding this comment. 🩺 Stability & Availability | 🟡 Minor | ⚡ Quick win 🧩 Analysis chain🏁 Script executed: #!/bin/bash
set -eu
rg -n --hidden --glob '!target' \
'target-cpu|x86-64-v3|avx2|cargo test|runs-on' \
.cargo .github 2>/dev/null || true
lscpu | rg -i 'Architecture|Flags' || trueRepository: AdaWorldAPI/ndarray Length of output: 149 🏁 Script executed: #!/bin/bash
set -eu
printf 'Tracked CI/config-related paths:\n'
git ls-files | rg '(^\.github/|^\.cargo/|Cargo\.toml$|Cargo\.lock$|^rust-toolchain\.toml$|\.gitlab-ci\.yml$|tox\.ini$|pyproject\.toml$|^README\.md$)' || true
printf '\nSearch targeted terms in small config/text files:\n'
rg -n --hidden --glob '!target' --glob '!*.lock' \
'target-cpu|x86-64-v3|x86_64|avx2|cargo test|cargo +|RUSTFLAGS|RUST_TEST_THREADS|RUST_BACKTRACE|runs-on:|runner:' \
.cargo .github Cargo.toml rust-toolchain.toml README.md 2>/dev/null || trueRepository: AdaWorldAPI/ndarray Length of output: 149 🏁 Script executed: #!/bin/bash
set -eu -o pipefail
printf 'Repository root files:\n'
find . -maxdepth 2 \( -type f -o -type d \) 2>/dev/null | sed 's#^\./##' | sort | head -200
printf '\nCI/config tracked paths:\n'
git ls-files 2>/dev/null | sort | rg '(^\.github/|^\.cargo/|^Cargo\.toml$|^Cargo\.lock$|^rust-toolchain\.toml$|^\.gitlab-ci\.yml$|^tox\.ini$|^pyproject\.toml$|^README\.md$|^scripts/|^build\.rs$)' || true
printf '\nTargeted term matches (config files only):\n'
git ls-files 2>/dev/null | rg '(^\.github/|^\.cargo/|^Cargo\.toml$|^rust-toolchain\.toml$|^README\.md$|^scripts/|^build\.rs$)' | while read -r f; do
echo "--- $f"
rg -n -i 'target-cpu|x86-64-v3|x86_64|avx2|cargo test|cargo \+|RUSTFLAGS|RUST_TEST_THREADS|RUST_BACKTRACE|runs-on:|runner:|actions/runner' "$f" 2>/dev/null || true
doneRepository: AdaWorldAPI/ndarray Length of output: 149 🏁 Script executed: #!/bin/bash
set -eu -o pipefail
printf 'pwd and quick directory count:\n'
pwd
find . -maxdepth 3 -type d -print 2>/dev/null | wc -l
find . -maxdepth 3 \( -type f -o -type d \) 2>/dev/null | head -200
printf '\nTracked CI/config paths:\n'
git ls-files 2>/dev/null | sort | head -500 | rg '(^\.github/|^\.cargo/|^Cargo\.toml$|^rust-toolchain\.toml$|^README\.md$|^scripts/|^build\.rs$)' || true
printf '\nTracked all files count/sample:\n'
git ls-files 2>/dev/null | wc -l
git ls-files 2>/dev/null | head -100Repository: AdaWorldAPI/ndarray Length of output: 5157 🏁 Script executed: #!/bin/bash
set -eu -o pipefail
printf 'Relevant CI/config files with targeted matches:\n'
for f in $(git ls-files 2>/dev/null | rg '(^\.github/|^\.cargo/config[^/]*\.toml$|^Cargo\.toml$|^rust-toolchain\.toml$|^README\.md$|^scripts/.*\.sh$|\.gitlab-ci\.yml$)' || true); do
p=$(printf '%s' "$f" | sed 's/^\.\///')
echo "--- $p"
rg -n -i 'target-cpu|x86-64-v3|x86_64|avx2|cargo test|cargo \+|RUSTFLAGS|RUST_TEST_THREADS|RUST_BACKTRACE|runs-on:|runner:|actions/runner|nightly|default-target|default_target|required-features|features=' "$p" 2>/dev/null || true
done
printf '\nCloned known files summaries:\n'
for f in Cargo.toml rust-toolchain.toml README.md .cargo/config.toml .cargo/config-native.toml .cargo/config-apple-m2.toml .cargo/config-pi5.toml .cargo/config-graviton.toml .cargo/config-avx512.toml scripts/all-tests.sh; do
if [ -f "$f" ]; then
echo "===== $f ====="
cat -n "$f" | sed -n '1,220p'
fi
doneRepository: AdaWorldAPI/ndarray Length of output: 149 🏁 Script executed: #!/bin/bash
set -eu -o pipefail
printf 'Cargo workspace configs:\n'
sed -n '1,180p' Cargo.toml || true
printf '\nRust toolchain:\n'
cat rust-toolchain.toml || true
printf '\nCI config files present:\n'
find .github .cargo scripts README.md Cargo.toml rust-toolchain.toml 2>/dev/null -type f | sort
printf '\nGitHub config snippets:\n'
for f in .github/workflows/*.yml .github/workflows/*.yaml; do
[ -f "$f" ] && { echo "===== $f ====="; rg -n -i 'target-cpu|x86-64-v3|x86_64|avx2|cargo test|cargo \+|RUSTFLAGS|runs-on:|runner:|actions/runner|nightly|default-target|features=' "$f" || true; }
done
printf '\nCargo config snippets:\n'
for f in .cargo/config.toml .cargo/config-native.toml .cargo/config-apple-m2.toml .cargo/config-pi5.toml .cargo/config-graviton.toml .cargo/config-avx512.toml .cargo/config-wasm.toml; do
[ -f "$f" ] && { echo "===== $f ====="; sed -n '1,200p' "$f"; }
doneRepository: AdaWorldAPI/ndarray Length of output: 149 🏁 Script executed: #!/bin/bash
set -eu -o pipefail
tmpdir=$(mktemp -d)
trap 'rm -rf "$tmpdir"' EXIT
printf 'Quick extract config snippets to temp:\n'
for f in Cargo.toml rust-toolchain.toml README.md .cargo/config.toml .cargo/config-native.toml .cargo/config-apple-m2.toml .cargo/config-pi5.toml .cargo/config-graviton.toml .cargo/config-avx512.toml .cargo/config-wasm.toml; do
if [ -f "$f" ]; then
echo "===== $f ====="
sed -n '1,220p' "$f" > "$tmpdir/$f" || true
rg -n -i 'target-cpu|x86-64-v3|x86_64|avx2|cargo test|cargo \+|RUSTFLAGS|RUST_TEST_THREADS|RUST_BACKTRACE|runs-on:|runner:|actions/runner|nightly|default-target|features=' "$tmpdir/$f" || true
fi
done
echo "===== GitHub workflows quick scan ====="
for f in .github/workflows/*.yml .github/workflows/*.yaml; do
[ -f "$f" ] && { echo "===== $f ====="; sed -n '1,220p' "$f"; }
done
printf '\nNative AVX512 config existence/key lines:\n'
wc -l .cargo/config-avx512.toml .cargo/config-native.toml 2>/dev/null || true
rg -n -i 'avx512|x86-64-v3|target-cpu|cargo' .cargo/config-avx512.toml .cargo/config-native.toml 2>/dev/null || trueRepository: AdaWorldAPI/ndarray Length of output: 149 🏁 Script executed: #!/bin/bash
set -eu -o pipefail
tmpdir=$(mktemp -d)
trap 'rm -rf "$tmpdir"' EXIT
printf 'Copying known config files before grep:\n'
for f in Cargo.toml rust-toolchain.toml README.md .cargo/config.toml .cargo/config-native.toml .cargo/config-apple-m2.toml .cargo/config-pi5.toml .cargo/config-graviton.toml .cargo/config-avx512.toml .cargo/config-wasm.toml; do
if [ -f "$f" ]; then
sed -p '1,220p' "$f" > "$tmpdir/$f" || true
echo "copied $f"
else
echo "missing $f"
fi
done
printf '\nMatches from temp copies:\n'
for f in "$tmpdir"/Cargo.toml "$tmpdir"/rust-toolchain.toml "$tmpdir"/README.md "$tmpdir"/.cargo/* "$tmpdir"/.github/workflows/*; do
if [ -f "$f" ]; then
echo "--- $f"
rg -n -i 'target-cpu|x86-64-v3|x86_64|avx2|cargo test|cargo \+|RUSTFLAGS|RUST_TEST_THREADS|RUST_BACKTRACE|runs-on:|runner:|actions/runner|nightly|default-target|features=' "$f" || true
fi
doneRepository: AdaWorldAPI/ndarray Length of output: 7485 Do not equate
🤖 Prompt for AI Agents |
||
|
|
||
| ## Provenance | ||
|
|
||
| Investigated 2026-07-28 after a lowering PR (closed unmerged, ndarray #261) | ||
| shipped ~700 lines of hand-written intrinsics on the premise that these | ||
| polyfills executed scalar code. They do not. The PR was closed on the | ||
| operator's `U32x8`-composition ruling; this investigation is the deliverable | ||
| the ticket originally asked for. | ||
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
🗄️ Data Integrity & Integration | 🟠 Major | ⚡ Quick win
Synchronize the closed status across both knowledge documents. The append-only closure leaves contradictory pending markers in the documentation.
.claude/knowledge/agnostic-surface-cpu-matrix.md#L101-L118: remove or annotate the remaining⏳cells and “Needs verification” legend..claude/knowledge/td-simd-tier-audit.md#L274-L290: update or mark the retained “Audit pending” entry as superseded.📍 Affects 2 files
.claude/knowledge/agnostic-surface-cpu-matrix.md#L101-L118(this comment).claude/knowledge/td-simd-tier-audit.md#L274-L290🤖 Prompt for AI Agents