perf: shard the once table behind read-write locks

[aisk]

Builds directly on #225 — its commit is included unchanged, and @Huliiiiii is co-author on the follow-up commit. This keeps the 64-shard table and replaces the per-shard Mutex with std::sync::RwLock, porting the reader path from #205: hits and duplicate waiters probe their shard under the read lock, initialized compute hits resolve to the value without touching the entry reference count, and only an absent key takes the shard write lock.

Benchmarks

Bare-metal Linux, AMD Ryzen 7 5700X (8C/16T), NixOS 26.11, kernel 7.2.0, rustc 1.96.0, t=8, --sample-count 100 --sample-size 10000, interleaved rounds ×3, medians of round-medians. Multipliers are speedups relative to main (cbe49ea); values below 1× are regressions.

scenario	main	this PR	#225	#205
compute_hit_same_key (read)	697 ns	258 ns (2.7×)	809 ns (0.86×)	166 ns (4.2×)*
get_hit_same_key (read)	513 ns	298 ns (1.7×)	593 ns (0.86×)	347 ns (1.5×)*
compute_hit_disjoint (64/1024, read)	803 / 804 ns	63 / 75 ns (12.7 / 10.7×)	102 / 97 ns (7.9 / 8.3×)	170 / 172 ns (4.7 / 4.7×)
get_hit_disjoint (64/1024, read)	662 / 653 ns	50 / 51 ns (13.4 / 12.9×)	86 / 78 ns (7.7 / 8.4×)	516 / 526 ns (1.3 / 1.2×)
compute_miss_churn (64/1024, write)	2.75 / 2.60 µs	236 / 252 ns (11.6 / 10.3×)	231 / 240 ns (11.9 / 10.8×)	4.0 / 3.9 µs (0.69 / 0.67×)
compute_mixed (64/1024, read+write)	2.00 / 1.95 µs	167 / 171 ns (12.0 / 11.4×)	178 / 179 ns (11.2 / 10.9×)	2.6 / 2.5 µs (0.77 / 0.78×)
work_same_key (write)	1.97 µs	1.91 µs (1.0×)	1.85 µs (1.1×)	2.09 µs (0.94×)
work_disjoint_churn (write)	2.85 µs	240 ns (11.9×)	236 ns (12.1×)	3.76 µs (0.76×)
coalesced (both, wait-bound)	~18.6 µs	~18.7 µs (1.0×)	~18.5 µs (1.0×)	~18.5 µs (1.0×)

The main and "this PR" columns are from one interleaved run; the #225 and #205 columns are from an interleaved run earlier the same day on the same machine and method.

* the same-key deltas between this PR and #205 are within this machine's bimodal variance (both sides alternate between ~150 ns and ~270 ns modes across runs).

This PR is the only variant of the three that regresses no scenario. The pattern: the reader path pays off on the read-heavy scenarios (2–13× over main, 2–3× over #225 on same-key hits), costs 2–5% vs #225 on the write-heavy ones (an exclusive RwLock acquisition is slightly pricier than a Mutex), and the wait-bound scenarios are indifferent to the locking scheme.

---
config:
  xyChart:
    height: 900
  themeVariables:
    xyChart:
      plotColorPalette: "#59a14f, #4e79a7, #f28e2b, #9e9e9e"
---
xychart-beta horizontal
    title "Speedup over main, t=8 (main = 1x, longer is better)"
    x-axis ["R cHitSame base", "R cHitSame 205", "R cHitSame 225", "R cHitSame 228", "R gHitSame base", "R gHitSame 205", "R gHitSame 225", "R gHitSame 228", "R cDisj base", "R cDisj 205", "R cDisj 225", "R cDisj 228", "R gDisj base", "R gDisj 205", "R gDisj 225", "R gDisj 228", "RW mixed base", "RW mixed 205", "RW mixed 225", "RW mixed 228", "W churn base", "W churn 205", "W churn 225", "W churn 228", "W workSame base", "W workSame 205", "W workSame 225", "W workSame 228", "W workChurn base", "W workChurn 205", "W workChurn 225", "W workChurn 228", "wait coalesced base", "wait coalesced 205", "wait coalesced 225", "wait coalesced 228"]
    y-axis "speedup (x)" 0 --> 14
    bar [0, 0, 0, 2.7, 0, 0, 0, 1.7, 0, 0, 0, 12.7, 0, 0, 0, 13.4, 0, 0, 0, 12.0, 0, 0, 0, 11.6, 0, 0, 0, 1.0, 0, 0, 0, 11.9, 0, 0, 0, 1.0]
    bar [0, 0, 0.86, 0, 0, 0, 0.86, 0, 0, 0, 7.9, 0, 0, 0, 7.7, 0, 0, 0, 11.2, 0, 0, 0, 11.9, 0, 0, 0, 1.1, 0, 0, 0, 12.1, 0, 0, 0, 1.0, 0]
    bar [0, 4.2, 0, 0, 0, 1.5, 0, 0, 0, 4.7, 0, 0, 0, 1.3, 0, 0, 0, 0.77, 0, 0, 0, 0.69, 0, 0, 0, 0.94, 0, 0, 0, 0.76, 0, 0, 0, 1.0, 0, 0]
    bar [1, 0, 0, 0, 1, 0, 0, 0, 1, 0, 0, 0, 1, 0, 0, 0, 1, 0, 0, 0, 1, 0, 0, 0, 1, 0, 0, 0, 1, 0, 0, 0, 1, 0, 0, 0]

Loading

Colors: gray = main (baseline, 1x) · orange = #205 (whole-table RwLock) · blue = #225 (Mutex shards) · green = this PR (RwLock shards, combo). Label prefixes: R = read-heavy, W = write-heavy, RW = mixed, wait = wait-bound. Disjoint/churn/mixed use the 64-entry variant.

Other changes

• Ports the cleanup regression tests and the Send-only-hasher trait assertions from perf: take the keyed table read lock on hits and duplicate waiters #205.
• Expands the cleanup_abandoned_entry comment with why the strong_count == 2 check stays sound when entries are cloned under read locks.
• CHANGELOG entries, including the S: Sync breaking change — which applies to any design that hashes outside the exclusive lock, perf: sharded OnceTable #225 included.

Which PR to merge

My honest ranking:

• perf: take the keyed table read lock on hits and duplicate waiters #205 should be closed regardless: it wins only the same-key hit paths and regresses the write-heavy scenarios on both machines measured (25–40% on churn).
• This PR, if the read/write-lock discipline is judged worth it: both halves of the design were already reviewed individually in perf: take the keyed table read lock on hits and duplicate waiters #205 and perf: sharded OnceTable #225, and the extra complexity over perf: sharded OnceTable #225 is the read-probe → write-insert double check plus a slightly longer cleanup argument.
• perf: sharded OnceTable #225 is the right choice if the maintainers prefer the minimal locking discipline (one exclusive lock per operation). Its only cost is same-key hits, and the reader path here can be layered on later.

One honest caveat: std::sync::RwLock readers are expensive under sustained same-key contention on macOS (see the M4 Max numbers in #205), and sharding does not spread same-key traffic, so that caveat carries over to this PR. Mutex shards (#225) do not have it.

🤖 Generated with Claude Code

[tisonkun]

I think a more significant comparision column is current main 🤣

[Huliiiiii]

In my tests, parking_lot::RwLock is faster than std::RwLock, and you don't need to open another PR. Changing Mutex to RwLock only requires a few lines of code.

[tisonkun]

parking_lot::RwLock

I tend not to use parking_lot::RwLock unless that is a super strong reason. The less dependencies the better.

[aisk]

@Huliiiiii My main purpose here was to run the benchmarks for verification. I support merging #225 first, or continuing this work directly on #225. Happy to close this PR either way.

[aisk]

Fair point. Added a current-main (cbe49ea) column to the table, from a fresh interleaved run: 2.7x on same-key compute hits, 1.7x on same-key get hits, 10-13x on disjoint/churn/mixed, parity on coalesced.

[tisonkun]

Feel free to continue any of your works @Huliiiiii @aisk

However, I found this direction goes too far in complexity for a case-by-case perf benefit.

I tend to keep the code simple and only chase for perf when it is a bottleneck or there is a clear neat win.

Sorry to catch too much attention here.

[aisk]

Agreed on keeping it simple. I support advancing #225: simplest of the three, 8-12x on miss/churn/disjoint on Linux, with same-key hits ~15% slower than main as the only cost. One note: the S: Sync bound applies to #225 as well, since it also hashes outside the lock. Happy to close this PR and #205. If the RwLock reader path turns out to be needed later, we can iterate on top of #225.