Repository navigation
v0.2.0: fix incorrect results, rewrite the kernels, and re-document - #1
Merged
Merged
Conversation
v0.1.1 returned a single repeated value for every input on any system with NumPy 2.x. The kernels were computing correct values; the binding discarded them. This release fixes that, replaces the two hot kernels, and rebuilds the packaging, tests, CI and documentation around them. Correctness Results were a constant. python/CMakeLists.txt pinned pybind11 v2.10.4, which predates NumPy 2.0's change to the array descriptor layout. Built against NumPy >= 2 every returned array had a stride of 0, so all elements aliased element 0. CMake now requires pybind11 >= 2.12. Every build targeted sm_52. project(LANGUAGES CUDA) seeds CMAKE_CUDA_ARCHITECTURES, so the detection function's if(DEFINED ...) guard always returned early -- an sm_86 A6000 ran Maxwell code. Detection now uses nvidia-smi, overridable via CULOF_CUDA_ARCHITECTURES or CUDAARCHS. normalize=True issued cudaMemcpyHostToDevice on a device pointer and called cudaHostRegister on device memory. Non-contiguous input silently gave wrong answers: the binding tested only strides[1] before passing the raw pointer, so X[::2] was read as contiguous. k > 32 raised cudaErrorInvalidValue; zero-variance features produced NaN under normalize=True; --use_fast_math contradicted the documented aim of matching scikit-learn; BUILD_TESTS=ON could not configure. Performance Distances are now one cuBLAS SGEMM via |a-b|^2 = |a|^2 + |b|^2 - 2a.b on centred data, replacing an elementwise kernel that accumulated in float64 -- a 1:64 rate on consumer Ampere. Centring leaves distances unchanged and keeps the identity from cancelling. k-NN is an exact radix select on the raw float bit patterns rather than a per-thread insertion sort. LOF needs the neighbour set and the k-distance, never their order, so selection suffices -- and the k-th key the select produces is the k-distance. No k limit, coalesced reads, O(n) per pass. The Gram-to-distance conversion is fused into the first pass and 11-bit digits cover 32 bits in three passes, so the tile is swept four times, not six. The n x n distance matrix is no longer materialised. Tiling keeps memory linear in n; n = 200,000 now runs where it previously needed 149 GiB. Measured on an RTX A6000 against a 24-thread i9-12900K, 8 features, k=20: 63x at n=1,000, 31x at n=50,000, 12x at n=200,000. API LOF is now a drop-in replacement for sklearn.neighbors.LocalOutlierFactor -- n_neighbors, contamination, fit_predict returning +/-1 labels, score_samples, negative_outlier_factor_, offset_. culof.lof(X, k) returns the LOF value itself. The previous set_k/set_threshold surface is gone, as is the pure-Python LOF class that __init__.py defined and then overwrote. Everything else Packaging moved to pyproject.toml with scikit-build-core; setup.py is gone and the version has one source. The Python package moved under src/ so tests exercise the installed artefact rather than the source tree. Removed the duplicated python/ package tree, which shipped a second copy of the tests and a top-level module named "python". Results are deterministic: the neighbour gather uses a block-wide prefix sum rather than atomics, so repeated runs are bit-identical. Added CI across CUDA 11.8/12.4 and Python 3.9-3.12, a C++ suite checking each stage against an independent double-precision CPU reference, and a Python suite checking scikit-learn parity plus one regression test per defect above. Benchmarks now verify agreement with scikit-learn in the same run that times them, and the figures are generated from that run's JSON -- the previous README, and two different figures, carried three mutually contradictory sets of numbers (6.75x, 4.6x, 1.57x).
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
v0.1.1 returned a single repeated value for every input on any system with
NumPy 2.x. The CUDA kernels were computing correct values; the binding threw
them away. This PR fixes that, replaces the two hot kernels, and rebuilds the
packaging, tests, CI and documentation around them.
Verified on an RTX A6000 (sm_86, CUDA 12.5): 45 Python tests and 11 C++ tests
pass,
ruffis clean, and a cleanpip installfrom the tree works.The headline bug
python/CMakeLists.txtpinned pybind11 v2.10.4, which predates NumPy 2.0'schange to the array descriptor layout. Built against NumPy ≥ 2, every returned
array had a stride of 0, so all elements aliased element 0:
The kernels were fine — the value being broadcast is the correct score for
point 0. CMake now requires pybind11 ≥ 2.12, and
test_output_is_not_a_constantguards the symptom.Other correctness fixes
project(LANGUAGES CUDA)seedsCMAKE_CUDA_ARCHITECTURES, so the detection function'sif(DEFINED ...)guard always returned early — an sm_86 A6000 ran Maxwell code.
normalize=Truewas undefined behaviour:cudaMemcpyHostToDeviceon adevice pointer, and
cudaHostRegisteron device memory.strides[1]before passing the raw pointer, soX[::2]was read ascontiguous.
k > 32raisedcudaErrorInvalidValue; zero-variance features produced NaNunder
normalize=True;--use_fast_mathcontradicted the documented aim ofmatching scikit-learn;
BUILD_TESTS=ONcould not configure.Performance
Distances are now a single cuBLAS
SGEMMvia‖a-b‖² = ‖a‖² + ‖b‖² - 2a·boncentred data, replacing an elementwise kernel that accumulated in float64 — a
1:64 rate on consumer Ampere.
k-NN is an exact radix select on the raw float bit patterns. LOF needs the
neighbour set and the k-distance, never their order, so selection suffices —
and the k-th key the select produces is the k-distance. No k limit, coalesced
reads, O(n) per pass. The Gram-to-distance conversion is fused into the first
pass and 11-bit digits cover 32 bits in three passes, so the tile is swept four
times rather than six.
The n×n matrix is no longer materialised; tiling keeps memory linear in n.
n = 200,000 now runs where it previously needed 149 GiB.
RTX A6000 vs 24-thread i9-12900K, k=20, best of 7, context init excluded.
API
LOFis now a genuine drop-in forsklearn.neighbors.LocalOutlierFactor—n_neighbors,contamination,fit_predictreturning ±1 labels,score_samples,negative_outlier_factor_,offset_. A test asserts label-for-labelequality with scikit-learn.
culof.lof(X, k)returns the LOF value itself.The previous
set_k/set_thresholdsurface is gone, as is the pure-PythonLOFclass that__init__.pydefined and then immediately overwrote with theC++ one — so the documented API never existed at runtime.
Accuracy
Agreement is measured, not asserted: median relative difference 4.5×10⁻⁷,
p99 2.4×10⁻⁶, identical top-1% rankings everywhere. The tail is points whose
k-th and (k+1)-th neighbours are closer than float32 can resolve, where the
choice of k-th neighbour is genuinely ambiguous; 0.27% of points, and it never
changed which points were flagged.
test_near_ties_are_the_only_disagreementpins this down rather than hiding it behind a loose tolerance.
Everything else
pyproject.toml+ scikit-build-core;setup.pygone; version has one source.src/so tests exercise the installed artefact, not the source tree.python/tree, which shipped a second copy of the tests and a top-level module namedpython.Note
The
.github/workflows/ci.ymlfile needs a token with theworkflowscope topush over HTTPS; this branch was pushed over SSH.