Rewrite X25519/X448 field arithmetic onto single 64-bit-limb representations with per-arch kernels - #181
Merged
Merged
Conversation
…esentation Replace the ten signed Int32 radix-2^25.5 limbs with one 5 x UInt64 radix-2^51 value record (TX25519Fe), used on every target. The field unit is arch-blind (no rep/arch ifdefs): multiply/square dispatch through the facade to a per-arch asm kernel and fall back to a portable Mul64-based scalar path (FORCE_SCALAR, and any arch without a kernel). Kernels (Include/Simd/Curve25519/X25519Field_Fe64_<arch>.inc): - x86-64, aarch64: hand-written, register-resident (plain mul/adc, mul/umulh), entered through the shared Proc-begin prologues. - i386: register-starved, so each 51x51 product is emulated as four 32x32 muls into a 128-bit stack accumulator; emitted by scripts/generators/gen-curve25519-fe64-simd.py. Consumers move to the record: X25519 ladder, X25519 key utilities and the full Ed25519 point arithmetic. Point coordinates become value fields, the in-place point formulas alias via PX25519Fe pointers, and the packed comb/wNAF tables become record arrays. Retire the now-dead radix-2^25.5 kernels.
…tation Port X448/Ed448 from the 16-limb radix-2^28 array field to a single 8-limb radix-2^56 value record (TX448Fe), mirroring the fe51 rewrite of X25519/Ed25519. The field unit is arch-blind; Mul/Sqr dispatch through the curve-field facade (TryMul448/TrySqr448) with a portable Pascal MulCore fallback. There is no fe56 asm kernel yet, so the backends return False and the Pascal path carries curve448. - ClpX448Field: record rewrite (Goldilocks reduction, ModOddInverse inverse via a byte bridge, single-candidate SqrtRatioVar, word Mul, 7-byte codec). - ClpX448 / ClpEd448 / ClpCurve448KeyUtilities: converted to the record API (point records with value fields, FillChar init, PX448Fe aliasing pointers in the point formulas, record-array precompute/lookup tables, base-point constants re-encoded to fe56). - Facade + X86/Arm backends gain Mul448/Sqr448 (False until the kernel). Fix a constant-time regression the 64-bit rewrite introduced: the fe51 and fe56 MulCore and word-Mul used the carry idiom "x := x + y; if x < y then Inc(hi)". Replace every carry site with the branchless "hi := hi + UInt64(Ord(x < y))" (compiles to setcc), so the software field multiply has no secret-dependent branches. Add x448, ed25519 and ed448 subjects to the constant-time harness (dudect + ctgrind, mirroring X25519): x448 covers the shared fe56 field, and the Ed sign subjects add the comb and scalar-arithmetic coverage the ladders do not exercise. Their secret is the seed (a poisonable byte buffer, hashed before use), so unlike the EC BigInteger scalars they taint-check cleanly in the ctgrind leg; the Ed dudect rows are clean-baseline, as there is no variable-time signer to pair as a control.
…d gate The gate script enumerates its run list explicitly, so the x448/ed25519/ ed448 targets added to GetValgrindTargets were compiled into CTValgrind but never invoked. Add them as clean subjects so the ctgrind leg poisons the curve448 field and the Ed sign paths on both the x86_64 and aarch64 scalar CI legs.
…d gate ScalarMultBaseEncoded validates [k]B with CheckPoint before encoding it as R / the public key A, then branches on the result. That value derives from the secret seed, so the ctgrind taint gate flags the branch. But [k]B is a prime-order base-point multiple - always a valid non-identity point - so the outcome is invariant, and the condition is a deterministic function of data the routine is about to publish (public-equivalent). Keep the fault/DFA defence check in the code and mask only this one branch, via a new narrow "public-output declassification" exception in ct.supp (scoped to the top frame; still forbidden for any non-published intermediate). Controls still fire with the file applied.
The gate only passed --suppressions when a grep guard decided ct.supp had "real entries"; that guard is the one step whose runner behaviour is not visible in the logs, and the ed25519/ed448 subjects were failing with "suppressed: 0 from 0" despite a committed, locally-verified suppression. Drop the guard and pass --suppressions whenever the file exists (an all-comments file just loads zero rules). Also make the gate self-diagnosing: echo the exact valgrind command and the ct.supp block count, and on a subject failure print the real stack frames (at/by 0x ...) plus the suppression block valgrind would generate, so a non-matching suppression can be fixed straight from the CI log instead of guessed at.
The gate compiled CTValgrind with -O3 and no -g, so the linker stripped the symbol table. Memcheck then reported every frame as '???', and the fun: suppressions in ct.supp could never match - the ed25519/ed448 sign subjects failed with "suppressed: 0 from 0" even though the suppressions were applied and correct (confirmed locally: a symbol-stripped binary reproduces '???' and zero matches; the same binary with symbols matches 2 from 2). Add -gl to the CTValgrind compile so the symbol table is kept and function- name suppressions resolve. ct.supp and the library are unchanged.
- x86-64: 256-byte stack frame, 8x8 schoolbook into 15 columns, Goldilocks fold (column k into k-4 and k-8), radix-2^56 carry. - aarch64: per-column register pairs (mul/umulh), extr for the 56-bit carry, caller-saved registers only. - i386: each 56x56 product as four 32x32 muls into 32-bit columns.
The value-type prime-field limbs are native-endian uint32 arrays, but the constant-time CIOS Montgomery scalar fallback read and wrote them as uint64 limbs via a memory reinterpret. That pairs correctly on little-endian, but on big-endian the two 32-bit halves of every 64-bit limb are swapped, so all prime-curve EC operations (ECDSA, Schnorr, MuSig2, ECDH, sum-of-multiplies) produced wrong results on PowerPC64. Read and write the 64-bit limbs by combining the native uint32 values arithmetically, through TBinaryPrimitives.LoadUInt32/StoreUInt32, and pack the modulus into the CIOS context the same way instead of a raw System.Move. The field operands become uint32-limb pointers; the context stays uint64. The assembly kernels are little-endian only and keep the uint64 view: on big-endian they are inactive and the fallback runs, so on little-endian the fallback and context are byte-identical to before and nothing changes.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Rewrites the X25519/Ed25519 and X448/Ed448 field arithmetic from their old
32-bit-limb representations onto a single 64-bit-limb representation per curve,
with hand/generated assembly kernels for x86-64, aarch64 and i386, and a
portable scalar fallback for every other target. Also adds the constant-time
coverage for these curves and a big-endian correctness fix for the value-type
Weierstrass field that the PowerPC64 CI target surfaced.
Representation change
(radix-2^51, 5x UInt64, prime 2^255-19).
fe56 (radix-2^56, 8x UInt64, Goldilocks 2^448-2^224-1).
The field unit is a plain value record with zero representation/arch
{$IFDEF}s.One byte layout serves every architecture; all arch specifics live below a
Try*facade. i386 keeps the same 64-bit layout and emulates the wide limbs with32x32 multiplies, so there is no second representation to maintain.
Kernels
Six new generator-owned
.inckernels (drift-gated), wired through the existingfacade/backend pattern:
X25519Field_Fe64_{x86_64,aarch64,i386}.inc(fe51 multiply/square)X448Field_Fe64_{x86_64,aarch64,i386}.inc(fe56 multiply/square)x86-64 and aarch64 fe51 are register-resident; the fe56 kernels and the i386
kernels are generated (register-starved / wide-limb emulation). Kernels gate only
on "not forced scalar" (plain integer asm, no SIMD feature dependency).
Constant-time
Adds x448, ed25519 and ed448 as subjects to both the dudect and the valgrind
(ctgrind) gates, alongside the existing x25519 subject that already covers the
shared field. Ed's secret is a contiguous seed that is hashed before use, so it
poisons cleanly like the other byte-secret subjects. The gate now builds the
valgrind binary with symbols and always applies the suppression file; the
always-true point-validity branch on the published
[k]Boutput is declassifiedthere.
Big-endian fix
The value-type Weierstrass CIOS Montgomery field (already on master) stored its
limbs as native-endian uint32 arrays but its scalar fallback reinterpreted them
as uint64 limbs. That is correct on little-endian, but on big-endian the two
32-bit halves of every 64-bit limb are swapped, so all prime-curve EC operations
produced wrong results on PowerPC64. The fallback now combines the native uint32
values arithmetically (via
TBinaryPrimitives.LoadUInt32/StoreUInt32) and packsthe modulus into the CIOS context the same way instead of
System.Move. Thelittle-endian asm path is byte-identical to before.