Skip to content

perf: widening 32x32 multiply in the vector kernels, extracted into VectorMath - #8

Merged
unsafePtr merged 20 commits into
masterfrom
perf/widening-multiply-32
Aug 7, 2026
Merged

perf: widening 32x32 multiply in the vector kernels, extracted into VectorMath#8
unsafePtr merged 20 commits into
masterfrom
perf/widening-multiply-32

Conversation

@unsafePtr

Copy link
Copy Markdown
Owner

Summary

Extracts the SIMD kernels shared by the 32/64-byte Bitcoin fast paths into a dedicated VectorMath class, and replaces the portable ulong multiply in them with an explicit widening 32x32 multiply. Adds arm64 support for that multiply, direct tests for the kernels, and a CI job that dumps arm64 codegen so the lowering can be checked on real hardware.

Why the widening multiply

Every operand in these kernels is under 2^32 — encode table entries and limbs are < 58^5, decode entries < 2^32, binary limbs are uint32. Neither AVX2 nor NEON has a 64x64 vector multiply, so a portable x * y on Vector*<ulong> has to be synthesised. Because the operands are narrow, a single widening instruction gives the identical answer:

portable x * y MultiplyWidening32
x64 / AVX2 8 instructions 1 (vpmuludq)
arm64 / NEON 8 instructions, 4 GPR round trips 3 (xtn, xtn, umull)

The JIT cannot make this substitution itself: it would need proof the operands are narrow, and they come from a runtime-built table and a span. Debug.Assert enforces the precondition, and it compiles out of release builds.

On AVX-512 hosts the portable form would lower to a single vpmullq, so the explicit intrinsic is not needed there — but vpmuludq is the cheaper instruction (1 uop vs 3), so the same code stays optimal across x64, AVX-512 and arm64.

Contents

  • VectorMathTensorDot, TensorMultiplyAdd, and the MultiplyWidening32 overloads they share. Keeps the MemoryMarshal / Unsafe / intrinsics surface in one auditable file.
  • arm64 lowering for the widening multiply (xtn + umull), after first trying uzp1.
  • .github/workflows/arm64-codegen-probe.yml — runs on perf/**, dumps VectorMath disassembly on Neoverse-N2 and Apple Silicon, and runs the full suite on arm64 (the publish workflow only covers master/PRs).
  • VectorMathTests — covers the kernels directly, including the Vector128 and scalar fallbacks.
  • Tests for fast-path rejection of values too large for 32/64 bytes.
  • Refactors: stateless helpers made static; fast decode paths return bool instead of a misleading Try prefix.

Verification

  • 141 tests green, including both long-running differential fuzz tests (BigInteger oracle plus SimpleBase cross-check, string and byte overloads).
  • Kernel results are bit-identical to the scalar form: the widening multiply wraps the same way, so no output changes.
  • arm64 codegen confirmed on two vendors via the probe job; Vector256 is inactive on arm64 (only the Vector128 length gate is emitted), so the 256-bit path is x64-only by construction.

Notes

🤖 Generated with Claude Code

unsafePtr added 20 commits July 28, 2026 00:24
Reading the Neoverse-N2 disassembly showed GetUpper() on the uzp1 result costs a
mov plus an ext, so the sequence was four instructions rather than the two the
instruction census suggested. Narrowing each operand with xtn packs the same lanes
in three, and gates on AdvSimd rather than AdvSimd.Arm64 since both intrinsics live
on the base class. x64 codegen is unchanged (vpmuludq, 206/175 bytes).
Runs the end-to-end benchmarks at the merge-base and at head on the same runner, on
arm64 and x64, and reports per-benchmark deltas. Significance comes from confidence
interval overlap rather than a raw percentage: comparing two runs of identical code
locally produced up to 8.6 percent drift, and one case cleared a 5 percent threshold
with nothing changed.
@unsafePtr
unsafePtr merged commit 40e8be8 into master Aug 7, 2026
4 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant