Skip to content

[MOD-17102] Add cosine-family support for COSINE_SIMILARITY#997

Open
rbs333 wants to merge 1 commit into
mainfrom
MOD-17012/cosine_similarity_metric
Open

[MOD-17102] Add cosine-family support for COSINE_SIMILARITY#997
rbs333 wants to merge 1 commit into
mainfrom
MOD-17012/cosine_similarity_metric

Conversation

@rbs333

@rbs333 rbs333 commented Jul 20, 2026

Copy link
Copy Markdown
Collaborator

Describe the changes in the pull request

Adds COSINE_SIMILARITY as a fourth vector metric (VecSimMetric_CosineSimilarity). It reuses the existing cosine-distance execution path end-to-end — no new spaces functions, preprocessors, ordering, heap, or comparator logic. A VecSimMetric_IsCosineFamily() helper is introduced so every place that special-cases VecSimMetric_Cosine (preprocessing/normalization, SVS factory, quant preprocessor, stored/query blob sizing, dist-func selection) treats the new metric identically. The public metric name and the exposed score/range semantics (similarity = 1 - distance) are translated at the API boundary in the RediSearch layer; this repo only adds the metric and routes it through the cosine internals.

Non-breaking: L2, IP, and COSINE behavior is unchanged, and the new enum value is appended last so existing serialized indexes keep their metric ordinals (L2=0, IP=1, Cosine=2).

Which issues this PR fixes

  1. MOD-17012
  2. SemanticCache threshold scale inconsistencies (async normalize_vector_distance and docstring) redis/redis-vl-python#407 (related — underlying cause)

Main objects this PR modified

  1. VecSimMetric enum + new VecSimMetric_IsCosineFamily() helper (vec_sim_common.h)
  2. Cosine-path routing: spaces.cpp (dist-func selection, all types incl. SQ8_FP16), preprocessors_factory.h, preprocessors.h (QuantPreprocessor), svs_factory.cpp, svs_utils.h
  3. Metric-aware sizing/stringify: vec_utils.cpp (VecSimMetric_ToString, VecSimParams_GetStoredDataSize), vec_sim.cpp (VecSimParams_GetQueryBlobSize)
  4. Python binding value (bindings.cpp)
  5. Unit tests: test_spaces.cpp, test_components.cpp

Mark if applicable

  • This PR introduces API changes
  • This PR introduces serialization changes

Note

Medium Risk
Touches metric enum and broad routing (dist funcs, preprocessing, SVS); behavior for existing L2/IP/Cosine should be unchanged but wrong routing would affect all cosine-family searches.

Overview
Adds VecSimMetric_CosineSimilarity as a fourth distance metric (appended after existing enum values) and VecSimMetric_IsCosineFamily() so cosine-specific behavior applies to both cosine metrics without duplicating checks.

VecSimMetric_CosineSimilarity follows the same internal path as VecSimMetric_Cosine: cosine preprocessors, normalization/IP shortcuts when vectors are already normalized, integral-vector norm padding in blob sizing, GetDistFunc dispatch (including SQ8 variants), SVS DistanceIP indexing, and QuantPreprocessor support. Stringify exposes COSINE_SIMILARITY; Python bindings export the new enum value. SVS drops a separate DistanceCosineSimilarity distance conversion template in favor of the shared IP conversion used for cosine metrics.

Unit tests cover the new metric in quant preprocessor suites and assert CosineSimilarity selects the same distance functions as Cosine.

Reviewed by Cursor Bugbot for commit b3bb365. Bugbot is set up for automated code reviews on this repo. Configure here.

@CLAassistant

Copy link
Copy Markdown

CLA assistant check
Thank you for your submission! We really appreciate it. Like many open source projects, we ask that you sign our Contributor License Agreement before we can accept your contribution.
You have signed the CLA already but the status is still pending? Let us recheck it.

@rbs333
rbs333 marked this pull request as ready for review July 20, 2026 20:13
@codecov

codecov Bot commented Jul 20, 2026

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 97.12%. Comparing base (dd6879e) to head (b3bb365).

Additional details and impacted files
@@            Coverage Diff             @@
##             main     #997      +/-   ##
==========================================
- Coverage   97.14%   97.12%   -0.03%     
==========================================
  Files         141      141              
  Lines        8245     8250       +5     
==========================================
+ Hits         8010     8013       +3     
- Misses        235      237       +2     

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@alonre24 alonre24 changed the title Add cosine-family support for COSINE_SIMILARITY [MOD-17102]Add cosine-family support for COSINE_SIMILARITY Jul 21, 2026
@alonre24 alonre24 changed the title [MOD-17102]Add cosine-family support for COSINE_SIMILARITY [MOD-17102] Add cosine-family support for COSINE_SIMILARITY Jul 21, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants