perf: widening 32x32 multiply in the vector kernels, extracted into VectorMath - #8
Merged
Conversation
Reading the Neoverse-N2 disassembly showed GetUpper() on the uzp1 result costs a mov plus an ext, so the sequence was four instructions rather than the two the instruction census suggested. Narrowing each operand with xtn packs the same lanes in three, and gates on AdvSimd rather than AdvSimd.Arm64 since both intrinsics live on the base class. x64 codegen is unchanged (vpmuludq, 206/175 bytes).
Runs the end-to-end benchmarks at the merge-base and at head on the same runner, on arm64 and x64, and reports per-benchmark deltas. Significance comes from confidence interval overlap rather than a raw percentage: comparing two runs of identical code locally produced up to 8.6 percent drift, and one case cleared a 5 percent threshold with nothing changed.
…h baseline" This reverts commit cd141ac.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Extracts the SIMD kernels shared by the 32/64-byte Bitcoin fast paths into a dedicated
VectorMathclass, and replaces the portableulongmultiply in them with an explicit widening 32x32 multiply. Adds arm64 support for that multiply, direct tests for the kernels, and a CI job that dumps arm64 codegen so the lowering can be checked on real hardware.Why the widening multiply
Every operand in these kernels is under 2^32 — encode table entries and limbs are < 58^5, decode entries < 2^32, binary limbs are
uint32. Neither AVX2 nor NEON has a 64x64 vector multiply, so a portablex * yonVector*<ulong>has to be synthesised. Because the operands are narrow, a single widening instruction gives the identical answer:x * yMultiplyWidening32vpmuludq)xtn,xtn,umull)The JIT cannot make this substitution itself: it would need proof the operands are narrow, and they come from a runtime-built table and a span.
Debug.Assertenforces the precondition, and it compiles out of release builds.On AVX-512 hosts the portable form would lower to a single
vpmullq, so the explicit intrinsic is not needed there — butvpmuludqis the cheaper instruction (1 uop vs 3), so the same code stays optimal across x64, AVX-512 and arm64.Contents
VectorMath—TensorDot,TensorMultiplyAdd, and theMultiplyWidening32overloads they share. Keeps theMemoryMarshal/Unsafe/ intrinsics surface in one auditable file.xtn+umull), after first tryinguzp1..github/workflows/arm64-codegen-probe.yml— runs onperf/**, dumpsVectorMathdisassembly on Neoverse-N2 and Apple Silicon, and runs the full suite on arm64 (the publish workflow only covers master/PRs).VectorMathTests— covers the kernels directly, including theVector128and scalar fallbacks.static; fast decode paths returnboolinstead of a misleadingTryprefix.Verification
Vector256is inactive on arm64 (only theVector128length gate is emitted), so the 256-bit path is x64-only by construction.Notes
refactor: slice to the fixed lengths in the bitcoin encode fast pathsreducesComputeBitcoin32FastRawfrom 917 to 639 bytes of codegen by letting the JIT fold ~50 bounds checks. Measured perf-neutral on both Zen 4 and Raptor Lake — it is a code-size change, not a speed-up, and the comment says so.IndexOutOfRangeExceptionwhen a leading zero reaches that boundary. Found by the fuzz test here and reported upstream as Base58 encoder truncates (and throws) for inputs encoding to exactly 128 characters — regression in 5.6.3 ssg/SimpleBase#87; the pin can move once that ships.🤖 Generated with Claude Code