Skip to content

Fix SymphonyQG inner-product query correction - #102

Merged
gouyt13 merged 1 commit into
mainfrom
fix/symqg-ip-query-correction
Sep 14, 2026
Merged

gouyt13 merged 1 commit into
mainfrom
fix/symqg-ip-query-correction

Conversation

@gouyt13

@gouyt13 gouyt13 commented Sep 14, 2026

Copy link
Copy Markdown
Member

Problem and change

SymphonyQG added the inner-product distance constant twice when estimating neighbors: the encoder's f_add already includes 1, while BatchQuery supplied the vertex distance 1 - q·c unchanged. With 64-dimensional constant vectors x = c = 0.125 and q = 0.0625, the neighbor estimate was 1.5 instead of 0.5, even though the residual is zero.

Make BatchQuery metric-aware and convert IP vertex distances to the query correction dist - 1 = -q·c, following IVF's query-side convention. Search and graph construction retain the original vertex distance separately for results. The encoder factors and estimator remain unchanged.

Regression coverage checks exact neighbor distances for zero and collinear residuals in both metrics, across dimensions 64, 1024, and 1088 (both FastScan accumulation paths). Python coverage verifies returned distances and save/load behavior. This fixes the internal distance offset; no recall or performance improvement is claimed.

Validation

  • Fresh Release C++ build with RABITQ_ENABLE_NATIVE_OPTIMIZATION=OFF: all 86 tests passed.
  • The regression failed with the old offset calculation and passes with the correction.
  • Rebuilt the Python wheel without changing dependencies, installed it in the existing environment, and verified the loaded extension matches the tested build.
  • python -m pytest tests/python/test_symqg.py tests/python/test_symqg_batch.py -q: 30 passed.
  • ./scripts/check-format.sh, ./scripts/check-python.sh, and git diff --check: passed.
  • Focused clang-tidy covering qg_test.cpp and included headers: passed using a GCC 9 compilation database. Clang 15 could not parse the machine's GCC 14 OpenMP headers; the compatible GCC 9 configuration resolved that tooling issue. Full pre-merge clang-tidy was not run.

Compatibility

BatchQuery gains an optional metric argument, defaulting to L2; existing two-argument L2 callers remain valid. IP callers must pass METRIC_IP, and g_add() then represents the estimator correction rather than the vertex result distance. Both SymphonyQG consumers are updated. No Python API, index-format, or dependency changes.

@gouyt13 gouyt13 self-assigned this Sep 14, 2026
@gouyt13
gouyt13 merged commit 03d8f1a into main Sep 14, 2026
18 checks passed
@gouyt13
gouyt13 deleted the fix/symqg-ip-query-correction branch September 14, 2026 11:37
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant