From a7ebe4e6b6c8824d4fcd9f590fed65ccfb9ec846 Mon Sep 17 00:00:00 2001 From: Aminsed <35748194+Aminsed@users.noreply.github.com> Date: Mon, 27 Jul 2026 22:50:04 -0400 Subject: [PATCH] Re-baseline every measured number onto one verified run Re-running benchmarks/benchmark.py on the same A6000 put the 20,000 and 50,000 rows about 10% below what the README claimed (30x and 29x measured against 33x and 31x). Small, and in the flattering direction, which is the direction that matters. Both tables now carry the numbers this run produced. The scaling exponents in docs/implementation.md did not reproduce at all -- n^2.01 and n^1.13 against a measured n^1.63 and n^1.39. The argument was sound; the fit range was the problem, and it was never written down. Fitting over all n mixes in the small-n region where the GPU is idle most of the time and the fit reports launch overhead rather than the algorithm. Over n >= 50,000, where both implementations are past their fixed overheads, the exponents are n^1.94 and n^1.31, so the ratio decays as n^-0.64 and predicts a 2.42x fall in speedup from 50k to 200k. The measured fall is 2.42x. The range is now stated in the text so the fit can be repeated. The d = 128 table needed no correction beyond rounding: measured 14/15/12/16/17 against a claimed 15/15/13/16/17, and the ratio really is flat -- it scales as n^0.05 across that range. The claim that ~15x is the steady-state hardware win holds. The one thing I could not verify is the assertion that both sides scale as n^1.9 at d = 128, since I measured the ratio and not the two curves separately, so the text now says what was measured. --- README.md | 8 ++++---- docs/implementation.md | 31 ++++++++++++++++--------------- 2 files changed, 20 insertions(+), 19 deletions(-) diff --git a/README.md b/README.md index 0dc3f07..a10bacc 100644 --- a/README.md +++ b/README.md @@ -31,10 +31,10 @@ answers. Changing the import is the whole migration. | samples | scikit-learn | cuLOF | speedup | |--------:|-------------:|------:|--------:| | 1,000 | 0.013 s | 0.0002 s | **63×** | -| 5,000 | 0.097 s | 0.0030 s | **32×** | -| 20,000 | 0.717 s | 0.0221 s | **33×** | -| 50,000 | 2.970 s | 0.0954 s | **31×** | -| 200,000 | 16.93 s | 1.411 s | **12×** | +| 5,000 | 0.096 s | 0.0030 s | **32×** | +| 20,000 | 0.659 s | 0.0221 s | **30×** | +| 50,000 | 2.723 s | 0.0953 s | **29×** | +| 200,000 | 16.66 s | 1.409 s | **12×** | RTX A6000 vs 24-thread i9-12900K, 8 features, k=20, best of 7 runs, one-time CUDA context initialisation excluded. Reproduce with: diff --git a/docs/implementation.md b/docs/implementation.md index 8535c94..088d90c 100644 --- a/docs/implementation.md +++ b/docs/implementation.md @@ -139,18 +139,19 @@ cuLOF computes all $n^2$ distances. scikit-learn uses a KD-tree or ball-tree in low dimensions, which never examines most pairs. These are **not the same algorithm**, and that governs how the comparison scales. -Fitting the measured timings in the range where both are past their fixed -overheads: +Fitting the measured timings over $n \geq 50{,}000$, where both are past +their fixed overheads — below that the GPU spends most of its time idle and the +fit reports the launch overhead rather than the algorithm: | | measured scaling | |---|---| -| cuLOF, brute force | $n^{2.01}$ | -| scikit-learn, KD-tree (d = 8) | $n^{1.13}$ | +| cuLOF, brute force | $n^{1.94}$ | +| scikit-learn, KD-tree (d = 8) | $n^{1.31}$ | -So the ratio decays as $n^{-0.7}$. A constant-factor hardware advantage, however +So the ratio decays as $n^{-0.64}$. A constant-factor hardware advantage, however large, always loses to a better asymptotic complexity eventually. Over -50,000 → 200,000 that predicts a $4^{0.7} = 2.6\times$ fall in speedup; the -measured fall is $2.59\times$. The narrowing is arithmetic, not a GPU running out +50,000 → 200,000 that predicts a $4^{0.64} = 2.42\times$ fall in speedup; the +measured fall is $2.42\times$. The narrowing is arithmetic, not a GPU running out of headroom. The prediction that follows is the useful one: it should be a **low-dimensional** @@ -160,14 +161,14 @@ sides are $O(n^2 d)$ and the ratio should stop decaying. It does: | n | speedup at d = 8 | speedup at d = 128 | |---:|---:|---:| -| 5,000 | 32× | 15× | -| 10,000 | 37× | 15× | -| 20,000 | 30× | 13× | -| 50,000 | 32× | 16× | -| 100,000 | **21×** | **17×** | - -At d = 128 both implementations scale as $n^{1.9}$ and the ratio is flat, even -rising slightly as the GPU saturates. +| 5,000 | 32× | 14× | +| 10,000 | 33× | 15× | +| 20,000 | 30× | 12× | +| 50,000 | 29× | 16× | +| 100,000 | **20×** | **17×** | + +At d = 128 the ratio is flat — fitted over this range it scales as $n^{0.05}$ — +and if anything rises slightly as the GPU saturates. Read together: **~15× is the honest steady-state hardware win**, sustained at any n once the algorithms match. The 30–60× seen at low dimensionality is larger