Build quantized QG using stored RaBitQ codes - #101
Merged
Merged
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Problem and change
Quantized QG retained the caller's original vectors for construction queries, pruning, and refinement even after encoding them. Construction now prepares a temporary source reconstruction from the existing RaBitQ codes and uses the existing estimator against target codes. The caller can release its input after the synchronous
QGBuilderconstructor finishes.This applies to candidate discovery, initial and fallback edges, pruning, and refinement. Entry-point selection scores codes against the centroid. Reverse candidates are rescored in destination-to-source direction because the estimates are asymmetric. The existing pruning thresholds/formulas remain in use; there is no dataset-sized reconstruction cache.
Validation
ctest --output-on-failure, native optimization OFF).git diff --check: passed.Python 3.11/3.14 CI remains to run; local Python testing used the existing 3.13 environment.
Representative benchmark
Current raw QG versus current code-only 4/8-bit QG construction. Each dataset samples 100,000 rows without replacement with NumPy
default_rng(42), sorted by original ID; all 1,000 queries are used with freshly recomputed exhaustive float64 top-10 ground truth. L2 for GIST/MSong/MXBAI, IP for Contriever. R=32, efConstruction=200, query ef=100, default three construction passes, fast encoding OFF, rotation seed 88, one thread pinned to CPU 0.Xeon Gold 6418H, GCC 14.3.0, C++17/O3, native optimization OFF, runtime AVX-512. One fresh-process build per case, sequential execution; query time is the median of five passes after warmup. Build time includes allocation/encoding/construction/refinement/neighbor packing, excluding input reading and serialization. Native graph initialization remains random, so small differences do not establish statistical significance. This compares storage modes in the new implementation, not before/after performance of this PR.
In these samples, 8-bit retains recall closer to raw and queries faster than 4-bit. The 4-bit representation has substantial recall losses; reducing memory does not guarantee better build or query time.
Compatibility
No public API, quantization, per-vector storage layout, index-format, dependency, or external-query scoring changes. Existing version-1 quantized files and legacy raw files retain their interpretation. Raw construction continues using its owned raw vectors. New quantized builds can produce different graphs and recall because construction now uses code-derived estimates throughout.