feat(index): add opt-in quantized vector refinement

[kamronis]

feat(index): add experimental quantized refinement results cache

Closes #8634.

Problem

Exact vector refinement reads original vectors from the dataset. On remote storage, these reads can dominate query
latency even though multi-bit RQ data is already available in the vector index.

Change

Add an explicit opt-in quantized refinement stage for flat IVF_RQ indexes.

When quantized_refine_factor is configured:

1. Overfetch factor * k candidates using Fast one-bit RQ scoring.
2. Preserve each candidate's IVF partition-local identity.
3. Rerank only those candidates using the index's Accurate stored RQ representation.
4. Return the best k results without reading original vectors for indexed candidates.

Exact refinement remains a separate optional final stage and can be combined with quantized refinement.

The option is exposed through:

• Rust: Scanner::quantized_refine
• Python: quantized_refine_factor
• Java: Query.Builder.setQuantizedRefineFactor

Scope

• Opt-in; default vector-search behavior is unchanged.
• Initially limited to flat IVF_RQ indexes.
• Unsupported index types and invalid factors return descriptive errors.

Validation

Added coverage for:

• RQ4 and RQ8 indexes
• L2, cosine, and dot-product metrics
• Multi-fragment datasets
• Quantized refinement followed by exact refinement
• Candidate identity and selected-candidate scoring
• NULL and non-finite vector cases
• Invalid and unsupported configurations
• Python and Java/JNI bindings

[lance-gatekeeper]

✅ Gate recommendation: approve.

The merge from main leaves the reviewed Fast-to-Accurate RQ patch unchanged. The newly merged batch-executor accessors are additive, and current-head verification confirms quantized refinement still routes through the per-query rerank while the expanded batch-KNN cases pass. Accurate RQ scoring, selective-prefilter handling, and separate exact refinement remain intact.

Please mark this PR with the breaking-change label.