Skip to content

Add the 1844th performance column, and what two runs of one commit say about the timings - #1099

Merged
Rafael-SOWNet merged 1 commit into
masterfrom
docs/the-1844th-performance-column
Aug 28, 2026
Merged

Add the 1844th performance column, and what two runs of one commit say about the timings#1099
Rafael-SOWNet merged 1 commit into
masterfrom
docs/the-1844th-performance-column

Conversation

@Rafael-SOWNet

Copy link
Copy Markdown
Collaborator

The performance column 2.4.0 needs, and a measurement that changes how the columns above should
be read.

The Kernel Benchmark was run twice on this same commit

Nothing changed between them.

run 1 run 2 apart
ParseEasy 12,593 ns 6,846 ns 46%
SimplifyEasy 197,016 ns 111,852 ns 43%
SolveHard 1,638,350,518 ns 887,253,553 ns 46%
RunEasy 50 ns 21 ns 57%
all eighteen 30–57%

Allocation over the same pair: SimplifyEasy 128,098 B against 128,096 B, SolveHard
1,446,783,288 B against 1,446,779,520 B — 0.03%. A thousandfold difference in
reproducibility, which is precisely the argument PerformanceGate already makes for gating
allocation and not time. It had not been measured on this workflow before.

So a timing difference between two columns of this file is evidence only above ~50%. The
1769th's write-up reads −7%, −10%, −15% as improvements and concludes "every row is faster or
flat"; none of those is distinguishable from the runner. Only its SolveEasy row survives —
2.5x with allocation moving 2.3x alongside — and it survives by the rule that write-up itself
states.

The table takes run 2; run 1 is in the prose so the choice is visible rather than quiet.

The exchange cost 13% of SimplifyEasy

Real, and measured where this file cannot measure — one desktop, both arms built and run the
same way, BenchmarkDotNet StdDev under 0.5%:

SimplifyEasy
1769th, before the rule sets became data 82,676 ns
after the exchange, before the node-type index 148,223 ns
after the index 94,904 ns
1843rd 93,732 ns

Every conversion was measured against the commit in front of it and every one came back free or
better. Nothing was measured against the start. The index recovered most of what the exchange
spent; most is not all.

Counted rather than guessed: 4,914 rule-set invocations at both commits — the work shape is
identical, so this is per-operation overhead, ~1.3 ns per invocation and ~14 ns per rule attempt.
InequalityEquality is 26% of it; InvertNegativePowers, a set with one rule, is 15%. Three
attempts at making dispatch cheaper (one-slot cache, eight-slot cache, array index instead of the
dictionary) were each measured and none moved it. About 40% is still unattributed and this
says so.

The lever, if wanted, is the invocation count rather than the dispatch — 4,914 for one
Simplify, of which 2,202 are the canonical order on every node.

Docs only; no code changes.

🤖 Generated with Claude Code

https://claude.ai/code/session_01Bjumi5K7fg8yx6UK1mZTQd

…y about the timings

The column for 2.4.0. Seventy-five commits from the 1769th, carrying the rule-set
exchange, Hensel lifting along an evaluation homomorphism, and four wrong answers found by
writing rules out as data.

**The Kernel Benchmark was run twice on this same commit, and the two runs are 30-57%
apart on every one of eighteen benchmarks.** Allocation over the same pair agrees to
0.03% -- `SimplifyEasy` 128,098 B against 128,096 B. That is the thousandfold difference
`PerformanceGate` already argues from, measured here on the workflow this file's columns
come from.

So a timing difference between two of these columns is evidence only above about 50%. The
1769th's write-up reads -7%, -10%, -15% as improvements and concludes "every row is faster
or flat"; none of those is distinguishable from the runner. Only its `SolveEasy` row
survives, and it survives because allocation moved 2.3x alongside the 2.5x -- which is the
rule that write-up itself states.

The table takes run 2 and run 1 is recorded in the prose, so the choice is visible.

**And the exchange cost 13% of `SimplifyEasy`, which this file cannot measure but a
desktop can.** 82,676 ns at the 1769th, 148,223 after the conversions, 94,904 after the
node-type index, 93,732 at the 1843rd -- one machine, both arms, standard deviation under
half a per cent. Every conversion was measured against the commit in front of it and each
came back free or better; nothing was measured against the start.

Counted rather than guessed: 4,914 rule-set invocations on this input at both commits, so
the work shape is identical and this is per-operation overhead -- about 1.3 ns per
invocation, 14 ns per rule attempt. `InequalityEquality` is 26% of it, `InvertNegativePowers`
15% from a single rule. Three ways of making the dispatch cheaper were measured and none
moved it; about 40% is unattributed. The lever is the invocation count, not the dispatch.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Bjumi5K7fg8yx6UK1mZTQd
@Rafael-SOWNet
Rafael-SOWNet merged commit de7189b into master Aug 28, 2026
31 checks passed
@Rafael-SOWNet
Rafael-SOWNet deleted the docs/the-1844th-performance-column branch August 28, 2026 12:05
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant