Skip to content

Build quantized QG using stored RaBitQ codes - #101

Merged
gouyt13 merged 1 commit into
mainfrom
qg-code-only-construction
Sep 13, 2026
Merged

gouyt13 merged 1 commit into
mainfrom
qg-code-only-construction

Conversation

@gouyt13

@gouyt13 gouyt13 commented Sep 13, 2026

Copy link
Copy Markdown
Member

Problem and change

Quantized QG retained the caller's original vectors for construction queries, pruning, and refinement even after encoding them. Construction now prepares a temporary source reconstruction from the existing RaBitQ codes and uses the existing estimator against target codes. The caller can release its input after the synchronous QGBuilder constructor finishes.

This applies to candidate discovery, initial and fallback edges, pruning, and refinement. Entry-point selection scores codes against the centroid. Reverse candidates are rescored in destination-to-source direction because the estimates are asymmetric. The existing pruning thresholds/formulas remain in use; there is no dataset-sized reconstruction cache.

Validation

  • Full portable Release C++ suite: 85/85 passed (ctest --output-on-failure, native optimization OFF).
  • Full ASan/UBSan suite with leak detection enabled: 85/85 passed.
  • Freshly rebuilt/installed wheel in the existing Python 3.13 environment: 122 pytest tests passed; installed extension bytes match the built wheel.
  • Installed CMake consumer: 1/1 passed. Release-automation unit tests: 21/21 passed.
  • Full clang-tidy analysis of core and bindings, full clang-tidy 22 include analysis, C++ formatting, Python formatting/lint, ShellCheck, strict MkDocs, and git diff --check: passed.
  • Regression coverage releases/poisons input before iterative construction, checks all final edge scores against a scalar double-precision estimator reference, verifies valid unique degree-32 neighbors, and verifies codes/factors remain unchanged. Covers 4/8 bits, L2/IP, padded dimensions, zero residuals and version-1 format/search round trips.
  • Previously saved raw, 4-bit and 8-bit GIST indexes return bit-identical IDs/distances for 100 queries using the new extension.

Python 3.11/3.14 CI remains to run; local Python testing used the existing 3.13 environment.

Representative benchmark

Current raw QG versus current code-only 4/8-bit QG construction. Each dataset samples 100,000 rows without replacement with NumPy default_rng(42), sorted by original ID; all 1,000 queries are used with freshly recomputed exhaustive float64 top-10 ground truth. L2 for GIST/MSong/MXBAI, IP for Contriever. R=32, efConstruction=200, query ef=100, default three construction passes, fast encoding OFF, rotation seed 88, one thread pinned to CPU 0.

Xeon Gold 6418H, GCC 14.3.0, C++17/O3, native optimization OFF, runtime AVX-512. One fresh-process build per case, sequential execution; query time is the median of five passes after warmup. Build time includes allocation/encoding/construction/refinement/neighbor packing, excluding input reading and serialization. Native graph initialization remains random, so small differences do not establish statistical significance. This compares storage modes in the new implementation, not before/after performance of this PR.

Dataset Storage Build (s) QPS Recall@10 Index (MiB)
gist Raw 101.58 12,645 95.81% 769.04
gist 4-bit 107.54 14,054 89.26% 449.38
gist 8-bit 101.56 15,346 94.73% 495.15
msong Raw 61.44 21,840 99.40% 367.74
msong 4-bit 58.17 23,197 89.79% 229.65
msong 8-bit 55.52 24,870 98.71% 251.01
contriever Raw 108.91 12,648 98.11% 622.56
contriever 4-bit 112.10 13,563 89.58% 366.98
contriever 8-bit 108.41 14,832 97.53% 403.60
mxbai Raw 117.78 15,083 99.28% 817.87
mxbai 4-bit 107.37 15,522 95.35% 476.84
mxbai 8-bit 96.54 17,224 99.10% 525.67

In these samples, 8-bit retains recall closer to raw and queries faster than 4-bit. The 4-bit representation has substantial recall losses; reducing memory does not guarantee better build or query time.

Compatibility

No public API, quantization, per-vector storage layout, index-format, dependency, or external-query scoring changes. Existing version-1 quantized files and legacy raw files retain their interpretation. Raw construction continues using its owned raw vectors. New quantized builds can produce different graphs and recall because construction now uses code-derived estimates throughout.

@gouyt13 gouyt13 self-assigned this Sep 13, 2026
@gouyt13
gouyt13 merged commit 0f068df into main Sep 13, 2026
20 checks passed
@gouyt13
gouyt13 deleted the qg-code-only-construction branch September 13, 2026 09:28
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant