Repository navigation
feat(bench): measure Apple M4 natively in the bench report - #435
Conversation
Quest-Claim: 5c6ac16208b97925
Adds kixtop (Apple M4, macOS) to BENCHMARKS.md from its quiet run at 93f91f4, and records why rind trails on short messages there: the M4 has DIT, which rind toggles around every call that handles a secret. A probe with DIT denied puts all but two of those rows within 10%. Rewrites the quest to the quiet desktop re-collect and kixtop's build cost, which both wait on the maintainer's hosts going quiet. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
|
Your OpenCode Go account is out of credit. Top it up, then re-trigger Pullfrog. OpenCode Go's response
|
|
Warning Review limit reachedYou've used all free OSS reviews for now. Wait for the free limit to reset to keep reviewing this public repository. Next included review available in 5 minutes. View limit details
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. |
kixelated
left a comment
There was a problem hiding this comment.
Automated review by review (OpenAI)
Reviewed head b44c0c6 against base e7570f6.
No actionable findings in the reviewed report/data changes. The M4 collection identifies commit 93f91f4, records 0.68 other busy CPUs against a limit of 1, and explicitly records the failed build-cost collection rather than inventing a measurement. Its 357 result IDs are unique and each estimate lies within its recorded interval. Recomputing the >10% comparisons from those results gives exactly the 19 DIT-listed operations plus the XChaCha20 32 B row, including the reported 12.62x maximum. The M4's sha3 feature excludes the !sha3 SHA-512/Ed25519 verification entries. The remaining quiet Arrow Lake collection and M4 build costs stay open in the quest.
Direction: the README now acknowledges Apple's short-message gaps. A separate held-DIT measurement remains useful for batch consumers, but keep it distinct from this per-call baseline and from the DIT-disabled diagnostic probe. Preserving the source commit and load qualification is important when newer kernel or dispatch changes land.
Limits: GitHub-only review of the report, raw JSON, gap entries, small renderer changes, verification predicates and quest updates. I did not rerun collection, the DIT-disabled probe, renderer freshness tests, CI, or proofs, and did not visually render the SVGs. The noise and causal explanations rely on the additional runs described in the PR, not independently reproduced measurements. This COMMENT is not merge approval.
…d rows Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

Works on quest/m1/bench-apple-m4.md: adds
kixtop(Apple M4, macOS) toBENCHMARKS.md, and rewrites the quest to what remains, a quietdesktopre-collect andkixtop's build cost, both waiting on the maintainer's hosts going quiet.Changes
benchmarks/data/kixtop.json: the second of two nativekixtopcollections, at93f91f48, which measured 0.68 other busy CPUs during the run (not noisy). The first, atcc2f73f6, ran under 4.36 other busy CPUs and most rows came out 30-40% slower, so it is not used. fix(bench-report): hold each host with quest gate --host, and measure macOS noise #399's macOS path worked unchanged; the ssh host key was already accepted.benchmarks/gaps.toml: twokixtopentries, both reasons:kixtopat the same commit with DIT denied (--cfg rind_cpu_disable,RIND_CPU_DISABLE_dit) put every one within 10% of the fastest library except AES header protection (1.2x, 0.6 ns) and HMAC-SHA-256 64 B (1.15x RustCrypto). Per-call DIT is the maintainer's decision from #387's findings, and #431'srind::cpu::Dit::hold()pays it once per batch.xchacha20poly1305/32B1.26x: noise. The probe's rerun and the first run both measured 1.04x.admin/bench-report: akixtophost note (in daily use, has DIT), and "in the Nix dev shell on Linux" becomes "in the Nix dev shell", since macOS uses it too since #399.README.md: the gaps example adds AES-GCM and short messages on Apple silicon, so "matches or beats both" stays true withkixtopin the headline.quest/m1/bench-apple-m4.md: rewritten to the remaining work, with aWaitingitem for the maintainer.quest mvrefuses on a claimed quest, so it keeps its path.quest/m3/bench-report-refresh.mdnow points here instead of at the quest.Verified table
Rechecked against
kixtop's features (aes, dit, neon, pmull, sha2, sha3); no change needed. Withsha3, rind runs the SHA-512 instructions, so the!sha3SHA-512 and Ed25519 entries correctly give no ✓. X25519, P-256 ECDH and ECDSA, and ChaCha20 header protection run the_altroutines and the same Rust their aarch64 entries cover, so they keep theirs.Why
desktopisn't herecollectrefuseddesktopthree times on 2026-11-07 (16.8, 4.7, and 5.3 of 24 CPUs busy, limit 2.4), all from other sessions' ungated builds. One attempt's host hold waited an hour: the globalquestondesktoppredates moq-dev/quest#526 and has no host-hold logic, so its gated commands were admitted past the hold. Bothkixtopruns' build costs were refused the same way (11.6 and 6.6 of 10 busy), so its build-cost table reads "Not measured".Choices made without asking
kixtop's quiet second run rather than averaging in the noisy first.kixtopnow and moveddesktopto the quest instead of holding the PR open for a quiet window that may not come soon.Impact
No public API or wire change. Report data, its renderer's host note, README wording, and quests only.
Validation
nix develop --command just check origin/main: pass on the pushed head.admin/bench-report render --check: pass.quest check --base origin/main: 70 documents ok.Follow-ups
rind::cpu::Dit::hold()on DIT cores in the report or nightly bench. feat(cpu): hold DIT across a batch of calls, putting M1 AES-GCM ahead of aws-lc-rs #431 skipped a held case because no report host had DIT;kixtopdoes, and MoQ will hold the guard, so the report shows a cost MoQ won't pay. Recommend: plan it.Models
Size: S
Implementer: Claude Code, claude-opus-5-5, high effort
Review
Rewritten by
quest reviewafter each round.01a1b43d-ccb7-7090-b831-b32b0ebf82f3, round 1The specified diff adds no cryptographic paths, the report and charts pass freshness validation, all eight rendering tests pass, and the missing M4 build costs are explicitly disclosed.