[MOD-14952] Support normalization in QuantPreprocessor

[dor-forer]

Describe the changes in the pull request

This PR is stacked on #999. Review only the incremental diff against dor-forer-MOD-14953-calc-distance-for-query.

Cherry-picks ARM's 027d5510b2bf7add6a45be9b0c07cda38c15b5ca from ARM-software/VectorSimilarity-for-Arm#2.

This PR adds an optional normalization-aware mode to QuantPreprocessor. It stores the mean-vector inner product as additional SQ8 metadata for normalized storage and queries, extends SQ8 metadata sizing, and shares a float-based quantization implementation to avoid duplicated logic.

Existing QuantPreprocessor configurations retain their current behavior. FP32 and FP16 tests cover normalization for L2 and inner-product metrics.

Which issues this PR fixes

1. MOD-14952

Main objects this PR modified

1. QuantPreprocessor
2. vecsim_types::sq8
3. Quantization component unit tests

Mark if applicable

• This PR introduces API changes
• This PR introduces serialization changes

Validation

• make format
• make check-format
• Built test_components and test_hnsw
• test_components: 30/30 passed
• test_hnsw with ROOT configured: 170/170 passed
• git diff --check dor-forer-MOD-14953-calc-distance-for-query...HEAD

---

Note

Medium Risk
Changes SQ8 blob layout and quantization math for a new API mode; default behavior is unchanged but any consumer must match metadata counts and distance kernels to the stacked PR.

Overview
Adds an optional WithNorm mode to QuantPreprocessor so SQ8 storage can quantize mean-centered vectors while queries stay in the original DataType space.

When WithNorm=true, storage quantization subtracts a supplied FP32 mean before min/max scaling; query preprocessing still copies raw values but records y_mean_ip = Σ(mean_i · y_i). Storage metadata gains x_mean_ip (inner product of the original vector with the mean before centering). vecsim_types::sq8 grows compile-time storage_metadata_count / query_metadata_count (and index helpers) by one float when normalization is enabled. Cosine is explicitly disallowed with WithNorm.

The quantization path is refactored through a shared quantize_impl on FP32 values (FP16 and norm paths widen/center into a temp buffer; plain FP32 without norm can still quantize in place). Default WithNorm=false keeps prior blob sizes and behavior.

Unit tests cover L2 and IP for float and float16, checking centered-quantization parity, blob sizes, and the new metadata slots.

^{Reviewed by Cursor Bugbot for commit d37077d. Bugbot is set up for automated code reviews on this repo. Configure here.}

[codecov]

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 97.11%. Comparing base (82f2462) to head (d37077d).

Additional details and impacted files

@@                             Coverage Diff                              @@
##           dor-forer-MOD-14953-calc-distance-for-query    #1000   +/-   ##
============================================================================
  Coverage                                        97.10%   97.11%           
============================================================================
  Files                                              141      141           
  Lines                                             8268     8296   +28     
============================================================================
+ Hits                                              8029     8057   +28     
  Misses                                             239      239           

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.