feat(gpu_hnsw): consume vanilla faiss GpuIndexHNSW + accurate int8 cosine#12
Open
devin-ai-integration[bot] wants to merge 3 commits into
Open
feat(gpu_hnsw): consume vanilla faiss GpuIndexHNSW + accurate int8 cosine#12devin-ai-integration[bot] wants to merge 3 commits into
devin-ai-integration[bot] wants to merge 3 commits into
Conversation
Author
🤖 Devin AI EngineerI'll be helping with this pull request! Here's what you should know: ✅ I will automatically:
Note: I can only respond to comments from users who have write access to this repository. ⚙️ Control Options:
|
![devin-ai-integration[bot]](https://avatars.githubusercontent.com/in/811515?s=60&v=4){kind=small}
…sine Rewire knowhere GPU HNSW onto the faiss-native faiss::gpu::GpuIndexHNSW (re-vendored faiss GPU tree), replacing the knowhere-specific GPU HNSW path. GPU_HNSW / GPU_HNSW_SQ index types are built from a vanilla faiss::IndexHNSW via the standard cloner and searched on GPU. - Vendored faiss GPU HNSW: unified layer-0 kernel (native int8 DP4A, warp- cooperative coalesced loads), native FP16/BF16 device storage, parallel bitonic-sort + merge-path merge, CPU-parity filtered search (deletes/TTL/ partition bitset), nq-chunked visited-bitmap VRAM bounding, and the review fixes (upload_bitset_if_needed in searchImpl_; layer-0-only nb_neighbors(1) guard). - Accurate int8 cosine: QT_8bit_direct_signed collapses L2-normalized vectors onto a few of 256 levels, so int8 cosine is re-encoded as fp16 at load (ToVanillaHnsw, gated on HasInverseL2Norms). int8 L2/IP keep direct_signed + DP4A. - GPU_HNSW load estimate (compact int8) populates Resource.maxMemoryCost; GetVectorByIds / raw-data retrieval unsupported (vectors kept on GPU). - Tests: FP16/BF16 deserialize + brute-force cosine oracle, CUDA fault injection, FP16 P1 regression, quantized-cosine recall gate. Clean single-commit re-creation off the v2 base (main 7964964); replaces the stacked gpu-hnsw-faiss-native branch. Signed-off-by: premal <premal@6sense.com> Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
devin-ai-integration
Bot
force-pushed
the
gpu-hnsw
branch
from
July 22, 2026 05:43
2a281eb to
97f933f
Compare
Mirror faiss #11 review fix into the vendored faiss GPU tree: select the native int8 DP4A layer-0 path on a per-search i8_queries_staged flag (set in GpuHnswSearchScratch::ensure from use_i8_queries) instead of sc.d_queries_i8 != nullptr. Pooled scratch slots keep d_queries_i8 allocated after an int8 search, so a later fp32-query search on the same slot could take the DP4A path against stale int8 query data; gating on the staged flag makes the fp32 fallback correct for reused slots. Also wraps vendored comment/source lines to 80 chars (no behavior change). Signed-off-by: premal <premal@6sense.com> Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
Mirror faiss gpu-hnsw dce385d9: searchImpl_ converts uint64 neighbor ids to idx_t labels on-device (convert_labels_kernel, UINT64_MAX -> -1) instead of a D2H -> host-convert -> H2D round-trip. Files byte-identical to the faiss repo post-change. Knowhere's production path uses searchHost/searchHostInt8, which were already round-trip-free, so this only affects the standard search() device-pointer path. Signed-off-by: premal <premal@6sense.com> Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Rewires knowhere GPU HNSW onto the faiss-native
faiss::gpu::GpuIndexHNSW(re-vendored faiss GPU tree), replacing the knowhere-specific GPU HNSW path.GPU_HNSW/GPU_HNSW_SQare built from a vanillafaiss::IndexHNSWvia the standard cloner and searched on GPU.This is a clean single-commit re-creation off the current
main(replaces the earlier stackedgpu-hnsw-faiss-nativebranch). Pairs with faissgpu-hnsw(GpuIndexHNSW). Consumed downstream by the milvusgpu-hnswpin.Vendored faiss GPU HNSW
Unified layer-0 kernel (native int8 DP4A, warp-cooperative coalesced loads), native FP16/BF16 device storage, parallel bitonic-sort + merge-path merge, CPU-parity filtered search (deletes/TTL/partition bitset), nq-chunked visited-bitmap VRAM bounding, plus the review fixes:
upload_bitset_if_needed(sc, sp, nq, stream)insearchImpl_(standardsearch()path).extract_hnsw_layers:maxM = (max_lv >= 1) ? nb_neighbors(1) : maxM0(no OOB read on a layer-0-only index).Accurate int8 cosine
QT_8bit_direct_signedis a fixedcode=x+128map, so L2-normalized (cosine) vectors — components ~1/sqrt(d)— collapse onto a few of 256 levels. int8-cosine is therefore re-encoded as fp16 at load:int8 L2/IP keep
direct_signed+ DP4A (no re-encode).Integration
GPU_HNSW load estimate (compact int8)populatesResource.maxMemoryCost.GetVectorByIds/ raw-data retrieval unsupported (vectors kept on GPU);StaticHasRawData=false.Tests
FP16/BF16 deserialize + brute-force cosine oracle, CUDA fault-injection, FP16 P1 regression, quantized-cosine recall gate (sq8 tracks CPU; fp16/bf16 vs oracle).
Link to Devin session: https://6sense.devinenterprise.com/sessions/55dab0bd33c346df99b5818b30059342
Requested by: @premal