perf: improve IVF-SQ training, encoding, and reader reuse

[JingsongLi]

Sparse IVF training partitions can learn narrow or constant scalar bounds that clip unseen residuals. IVF-SQ also repeats residual allocations, list transposition work, ID decoding, and query heap setup. This change pools residual bounds across the training sample and reduces build and repeated-query overhead while preserving the IVSQ v1 file layout.

• Train residual extrema with parallel partition reductions and encode directly into SQ8 output using precomputed scales and NEON/AVX2 conversion.
• Transpose serialized lists in parallel within bounded 16 MiB batches.
• Reuse one heap per batch query and apply a conservative partial-distance cutoff to L2 blocks.
• Cache decoded partitions in a FIFO charged to the existing reader memory budget, including resident metadata and cache bookkeeping. Cache initialization is encapsulated in reader construction, with the unified reader forwarding its existing header and options. Zero budget disables retention; direct IVFSQIndexReader::open remains uncached.
• Add behavioral coverage and update API, performance, index-selection, and benchmark reproduction documentation.

The version, required flags, blocked-code layout, and delta-varint IDs are unchanged. Existing files keep their recorded quantizers. Newly built files store pooled bounds in the existing per-list metadata; the pre-change reader at 8dcabf2 successfully reads them. Rebuilding obtains the new training bounds; reader caching and scan optimizations require the new reader.

Measured results

Three-run native ann_bench medians on Apple M4 Pro (12 CPU cores, 48 GiB RAM), Rust 1.95 release builds, eight workers, 1,024 partitions, 64 probes, Top-10, 65,536 training rows, and 1,000 held-out queries. Values show baseline 8dcabf2 → current, using the same input files and settings. GloVe is L2-normalized.

Corpus	Index build (ms)	Query P95 (µs)	Batch QPS	Recall@10
SIFT1M	934 → 886	840 → 298	6,417 → 8,987	0.8626 → 0.9811
GIST1M	6,404 → 5,850	4,386 → 1,828	899 → 998	0.8576 → 0.9400
GloVe-100	849 → 797	739 → 282	6,988 → 10,009	0.8036 → 0.8760

Build time includes training, encoding, and serialization. The native benchmark uses a 4 GiB reader budget: sequential queries can reuse partitions loaded earlier in the sweep; batch timing uses a separate fresh reader and includes payload reads and cache insertion. The baseline reader does not cache SQ partitions. File sizes are unchanged.

These measurements cover a warm local filesystem cache. Cold storage, object stores, other architectures, and other distributions need separate measurement. Pooled bounds can lose resolution on extreme-outlier data.

See the IVF-SQ benchmark results and reproduction guide.

Validation

• cargo test --workspace: 510 passed, 2 intentionally ignored, including v1 golden fixtures.
• Python bindings: 28 passed.
• Format, workspace Clippy with warnings denied, and license-header checks passed.
• Reader entry-point coverage verifies default, zero, insufficient, and sufficient cache budgets; repeated searches preserve results and expected I/O, and initialization reads the header and resident metadata exactly once.
• Documentation structure, local links, and benchmark values were checked.
• The pre-change reader opened newly generated SIFT, GIST, and GloVe files and completed 1,000 single queries plus batch search per corpus; recall differed from the new reader by at most 0.0002.
• x86_64 build and 33 SQ tests passed under Rosetta. AVX2 compiled, but Rosetta did not expose AVX2/FMA; native x86 runtime coverage remains for CI.

[leaves12138]

Reviewed 6603e4e73edc51575f7859e73492f90613e1e3c0 against main at 8dcabf208c99707eab3938af0bcc40530fdb4cad. No blocking findings.

I checked the pooled residual-bound training, fused SQ8 encoding and SIMD packing/tails, conservative L2 cutoff propagation, query-heap reuse, decoded-partition FIFO and memory accounting, cache misses/error recovery, and the unchanged IVSQ v1 serialization layout.

Validation on native x86_64 with AVX2/FMA available, using Rust 1.94.1:

• cargo test --workspace --locked: 510 passed, 2 intentionally ignored, including v1 golden-format fixtures and the reader-cache/filter/probe-range coverage.
• cargo fmt --all -- --check: passed.
• cargo clippy --workspace --all-targets --locked -- -D warnings: passed.
• Two additional local randomized differential tests passed: fused residual encoding matches the existing encoder across non-unit per-dimension ranges, clipping, reordered/repeated rows, and SIMD/tail dimensions; finite-cutoff native blocked scans preserve every competitive distance compared with the infinite-cutoff scan, including partial blocks and equality boundaries.

The additional tests exercise native AVX2 rather than only compiling that path. All current PR CI checks are also successful. I did not independently rerun the Apple M4 benchmark measurements, native NEON execution, or cold/object-store workloads; the documented qualification of those results is appropriate. LGTM.