Rogue SPR: score single-leaf candidates from parent split overlaps

[ms609-agent]

Speeds up spr_rogue_scores() (#174) by scoring most single-leaf candidates straight from the parent pair's split overlaps.

What I measured first

Before changing anything I instrumented a release build (rdtsc). The cost-matrix fill took 35% of runtime at 80 tips and 54% at 150. KeepAndReduce, GetSplits and FirstMatchingSplit together took 46% and 31%.

Popcount is not what makes the fill slow. In a standalone kernel with 1–3 64-bit bins, replacing the popcount with a lookup of parent counts saves about 1 cycle in 14.5 per cell. The time goes on the per-cell log arithmetic. So making the quadrant counts O(1), on its own, would not have helped. The gains come from two other places:

1. A cheaper cell formula (every bounded MCI call). Each cell's information is now Σ_q f(n_q) + row_term + col_term, where f(k) = k log₂ k comes from a table and the row and column terms are computed once per row and column. In the kernel this drops a cell from 14.5 to 8.5 cycles, and costs agree to within 1 unit.
2. Skipping the rebuild (SingleDrop, mutual_clustering_bounded_counts()). This applies to 96% of candidates on random pairs and 84% on 15-move pairs. It covers any leaf other than leaf 1 whose removal (a) makes nothing reducible and (b) makes no pair of splits identical. For those, the child cost matrix is the parent overlaps minus the leaf's membership, so KeepAndReduce, GetSplits, FirstMatchingSplit and the price hash are all skipped.
   • For (a), the cherry, root and chain tests of reduce_trees() are re-run on parent ledgers with the leaf removed.
   • For (b), the parent split pairs that differ in that leaf alone are found once per call.
   • Exactly one split vanishes from each tree: the row of the leaf's parent node, or of its sibling when the parent is the root's child.

Benchmarks

Two clean release installs (main, and this branch), run as interleaved separate Rscript processes, min of 9. I did not use compare-ab.R. SPR checksums were identical across builds.

Workload	main (s)	this PR (s)	Change
30 tips, random	0.45	0.20	−56%
30 tips, depth 2	3.73	3.60	−3%
80 tips, random	2.56	1.00	−61%
150 tips, random	7.97	3.19	−60%
80 tips, 15 moves	0.30	0.14	−53%
150 tips, 30 moves	1.30	0.59	−55%
80 tips, near-identical	0.04	0.03	(noise)

Depth 2 gains little because pair candidates still rebuild their trees.

The cell formula on its own gave −6% to −16% at 80 and 150 tips; skipping the rebuild accounts for the rest.

Correctness

• Same build, with and without SingleDrop: I replayed 11,064 spr_rogue_scores() calls (13,833 score vectors, 2.9 M candidates; 8–160 leaves, random and near-identical pairs, depths 1 and 2). Argmin sets were identical in every vector, and scores differed by at most 2.8e-14.
• Reduction test: it agreed with KeepAndReduce() (no extra leaf dropped, no shared split) on all 589,686 single-leaf candidates.
• Against main: the cell sums now add terms in a different order, so scores move by up to 4e-13 bits. As a result:
  • 5 of the 13,833 score vectors pick a different argmin. All five are exact ties (within 1e-12) that main broke by rounding noise.
  • 1 of 410 SPRDist() results changed, on an 8-leaf pair with such a tie.
  • Three tolerance = 0 checks of .BoundedCID() against ClusteringInfoDist() are relaxed to 1e-12.
• Narrower guard: the counts path leaves out the entropy bound that the bounded MCI applies first. It never fired in about 600,000 candidates.

Tests

• A new test compares the fast and rebuild paths on 60 random and near-identical pairs at depths 1 and 2. It also covers a fixed pair that triggers the chain rule, trees that share a split, and trees not rooted on leaf 1.
• covr: all new C++ lines are covered. The plain-lap() fallback is marked #nocov; it runs only above about 16k leaves.
• The full testthat suite passes locally, and spelling::spell_check_package() is clean.

Findings are recorded in .AGENTS/memory/Optimizations.md and dev/rogue_mci_bound_findings.md.

🤖 Generated with Claude Code

[github-actions]

Performance benchmark results

Call	Status	Change	Time (ms)
ClusteringInfoDistance(tr280)	⚪ NSD	0.88%	7.41, 7.37 → 7.32, 7.33
ClusteringInfoDistance(tr50)	⚪ NSD	0.06%	5.16, 5.11 → 5.15, 5.11
LAPJV(test2000)	⚪ NSD	2.03%	86.5, 88 → 85.7, 84.9
LAPJV(test40)	⚪ NSD	0.07%	0.0143, 0.0143 → 0.0144, 0.0143
LAPJV(test400)	⚪ NSD	-0.12%	2.58, 2.57 → 2.57, 2.58
MutualClusteringInfo(tr280)	⚪ NSD	-1.88%	15.8, 16.1 → 16.3, 16.1
MutualClusteringInfo(tr50)	⚪ NSD	-3.64%	14.9, 15.7 → 15.9, 15.7
NNIMaxStep(2000, normalize = TRUE)	⚪ NSD	0.9%	0.145, 0.147 → 0.145, 0.145
NNIMaxStep(300, normalize = TRUE)	⚪ NSD	-0.97%	0.0237, 0.0238 → 0.0239, 0.024
NNIMaxStep(300)	⚪ NSD	1.79%	0.00665, 0.00672 → 0.00666, 0.00649
PathDist(postTrees)	⚪ NSD	0.27%	1.89, 1.88 → 1.87, 1.89
PhylogeneticInfoDistance(tr180)	⚪ NSD	-0.27%	46.2, 46.5 → 46.5, 46.5
PhylogeneticInfoDistance(tr50)	⚪ NSD	0.26%	32.9, 32.7 → 32.8, 32.6
RobinsonFoulds(tr280)	⚪ NSD	1.04%	3.85, 3.82 → 3.81, 3.79
RobinsonFoulds(tr280)	⚪ NSD	0.19%	3.58, 3.59 → 3.59, 3.57
RobinsonFoulds(tr50)	⚪ NSD	1.08%	3.55, 3.55 → 3.51, 3.51

[codecov]

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 97.67%. Comparing base (9f3455f) to head (7339e46).

Additional details and impacted files

@@            Coverage Diff             @@
##             main     #223      +/-   ##
==========================================
+ Coverage   97.57%   97.67%   +0.10%     
==========================================
  Files          60       60              
  Lines        6056     6328     +272     
==========================================
+ Hits         5909     6181     +272     
  Misses        147      147              

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.