Added PQ INT8 quantizer in Turbo by zzlin237 · Pull Request #554 · alibaba/zvec

zzlin237 · 2026-07-07T08:17:20Z

The basic PQ has been implemented, but its QPS requires further optimization. The recall test results for the current implementation are as follows:

HNSW

Configuration:

M: 50
efConstruction: 500
efSearch: 400
topk: 50

L2 Metric

Datasets	Quantizer	Recall(%)	Refiner Recall (%)
Sift	kNone	100	-
Sift	kPQ + int8 + `64`	92.02	100
Sift	kPQ + int8 + `32`	78.44	99.98
Sift	kPQ + int8 + `16`	63.68	98.82
Gist	kNone	98.576	-
Gist	kPQ + int8 + `480`	93.212	98.668
Gist	kPQ + int8 + `240`	76.676	98.688
Gist	kPQ + int8 + `120`	54.40	94.45

The results are generally consistent with previous tests. Under the int8 condition, using one subquantizer per 4 dimensions yields the best performance.

Cosine Metric

Option 1: Normalization + IP Metric

Option 2: Normalization + L2 Metric

Datasets	Quantizer	Recall(%)	Refiner Recall (%)
Cohere	kPQ + int8 + `192` + Option 2	86.03	99.73
Openai	kPQ + int8 + `384` + Option 2	87.442	99.76

The performance of both options is nearly identical. Option 2 was ultimately adopted because the L2 metric also supports operations specific to L2, such as decentralization.

Inner Product Metric

Option 1: L2 for k-means + L2 encoding + IP for SDC calculation + IP for ADC calculation

Option 2: L2 for k-means + L2 encoding + L2 for SDC calculation + IP for ADC calculation |<---- faiss approach

Option 3: L2 for k-means + IP encoding + IP for SDC calculation + IP for ADC calculation

Failed approach: Due to mismatched encoding methods, recall dropped significantly.

Option 4: IP for k-means(sp) + IP encoding + IP for SDC calculation + IP for ADC calculation |<---- ivf porting approach

Failed approach: 1) Using IP in k-means leads to collapse, resulting in only one cluster containing points while all others remain empty; 2) Segmented clustering is unsuitable for spherical data. PQ performs segmentation before clustering. Even if the original vectors are normalized onto the unit sphere, their individual segments will no longer lie on the sphere.

Datasets	Quantizer	Recall(%)	Refiner Recall (%)
Cohere	kPQ + int8 + `192` + Option 1	79.904	99.658
Openai	kPQ + int8 + `384` + Option 1	82.41	99.936
Cohere	kPQ + int8 + `192` + Option 2	79.532	98.062
Openai	kPQ + int8 + `384` + Option 2	82.266	99.65

Option 1 performs slightly better and is more semantically coherent: L2 is used for encoding, while IP is used for distance calculation.

Decentralization

Datasets	Quantizer	Recall(%)	Refiner Recall (%)
Sift + L2	kPQ + int8 + `32`	78.44	99.98
Sift + L2	kPQ + int8 + `32` + decentralization	78.56	99.96
Cohere + Cosine	kPQ + int8 + `192`	86.03	99.73
Cohere + Cosine	kPQ + int8 + `192` + decentralization	86.088	99.738
Openai + Cosine	kPQ + int8 + `384`	87.54	99.78
Openai + Cosine	kPQ + int8 + `384` + decentralization	87.68	99.78

Decentralization does not yield significant improvements, but support for it is retained nonetheless.

IVF

Configuration:

centroid_count: 128
nprobe: 16
topk: 50
ScaleFactor: 4

Option 1: Build codebooks using residuals, with each cluster assigned an independent codebook.

Option 2: Build codebooks using residuals, with all clusters sharing a single codebook. ｜<--- faiss approach

L2 Metric

Datasets	Quantizer	Recall(%)	Refiner Recall (%)
Sift	kNone	99.7	-
Sift	IVFPQ + `int8` + `32` + Option 1	86.18	99.54
Sift	IVFPQ + `int8` + `32` + Option 2	78.52	99.44
Gist	kNone	96.412	-
Gist	IVFPQ + `int8` + `240` + Option 1	80.718	96.468
Gist	IVFPQ + `int8` + `240` + Option 2	77.49	96.244

Cosine Metric

Datasets	Quantizer	Recall(%)	Refiner Recall (%)
Cohere	kNone	99.576	-
Cohere	IVFPQ + `int8` + `192` + Option 1	85.174	94.37
Cohere	IVFPQ + `int8` + `192` + Option 2	83.916	94.09
Openai	kNone	96.024	-
Openai	IVFPQ + `int8` + `384` + Option 1	87.218	96.196
Openai	IVFPQ + `int8` + `384` + Option 2	83.916	94.09

Inner Product Metric

Datasets	Quantizer	Recall(%)	Refiner Recall (%)
Cohere	kNone	87.632	-
Cohere	IVFPQ + `int8` + `192` + Option 1	81.43	95.696
Cohere	IVFPQ + `int8` + `192` + Option 2	79.74	95.77

The original IVFPQ+IP implementation uses MIPS, which results in lower recall.

Although Option 2 shows a slight decrease in Recall compared to Option 1, it significantly reduces training time and storage requirements while maintaining comparable Refiner Recall. Therefore, Option 2 was ultimately adopted.

…efactor/turbo_preprocessor

…rocessor # Conflicts: # .gitignore

…o refactor/turbo_pq

Copilot

Copilot was unable to review this pull request because the user who requested the review has reached their quota limit.

Copilot

Copilot was unable to review this pull request because the user who requested the review has reached their quota limit.

Copilot

Copilot was unable to review this pull request because the user who requested the review has reached their quota limit.

Copilot

Copilot was unable to review this pull request because the user who requested the review is ineligible. To be eligible to request a review, you need a paid Copilot license, or your organization must enable Copilot code review.

Copilot

Copilot was unable to review this pull request because the user who requested the review is ineligible. To be eligible to request a review, you need a paid Copilot license, or your organization must enable Copilot code review.

Copilot

Copilot was unable to review this pull request because the user who requested the review has reached their quota limit.

Copilot

Copilot was unable to review this pull request because the user who requested the review has reached their quota limit.

feihongxu0824 · 2026-07-28T03:35:44Z

+//! via quantize_query().  Distance between a PQ code and a
+//! query uses ADC (LUT look-up); distance between two PQ codes uses
+//! SDC (centroid-to-centroid distance table).
+class PqInt8Quantizer : public Quantizer {


PqInt8Quantizer 这个命名严谨吗？我理解只是每个子量化使用8bit code

预计一共实现PqInt8Quantizer，PqInt4Quantizer和PqFastQuantizer三个量化器，对齐计划中的其他量化器取名：

RecordInt8Quantizer，RecordInt4Quantizer

PqInt4Quantizer和PqFastQuantizer是什么区别？

feihongxu0824 · 2026-07-28T03:36:59Z

+//! SDC (centroid-to-centroid distance table).
+class PqInt8Quantizer : public Quantizer {
+ public:
+  PqInt8Quantizer() {


为啥优先支持8bit？

除了支持8bit，后面还会支持其他数量的bit吗？

当前的HNSW中最适用的就是8bit，比如faiss支持8bit和16bit，vsag支持4bit和8bit

后续会支持其他数量的bit，由于把4bit也一起改了会导致一次性更改的文件太多，不太好review，计划是分两次pr提交

好奇问问为什么HNSW中最适用的就是8bit？

Copilot

Copilot was unable to review this pull request because the user who requested the review has reached their quota limit.

# Conflicts: # src/include/zvec/core/interface/index_param.h

Copilot

Copilot was unable to review this pull request because the user who requested the review has reached their quota limit.

richyreachy · 2026-07-30T11:44:37Z

    return DistanceImpl{};
  }

+  virtual DistanceImpl sym_distance(const void *query,


in quantizer base class, calc_distance_dp_dp is used for calculating datapoint distance. is there any need to define a new interface for symmetric distance? or can define it within pq quantizer then be exposed as calc_distance_dp_dp?

richyreachy · 2026-07-30T11:56:05Z

+                          const void *dist_table_v, size_t num_chunk,
+                          float *out) {
+  constexpr size_t kNumCentroids = 256;
+  constexpr size_t kTablePerSub = kNumCentroids * kNumCentroids;  // 65536


kTablePerSub，我们改成chunk吧这里，和外面统一

richyreachy · 2026-07-30T11:57:45Z

+}
+
+// ---------------------------------------------------------------------------
+// Simple Lloyd's KMeans for one sub-quantizer (templated on data type T).


richyreachy · 2026-07-30T12:00:59Z

+
+void PqInt8Quantizer::quantize_data(const void *input, void *output) const {
+  uint8_t *code = reinterpret_cast<uint8_t *>(output);
+  const uint32_t elem_sz = element_size();


richyreachy and others added 30 commits June 29, 2026 19:47

refactor: add quantizer and distance

b1313a4

fix: add turo quantizer

45d9b54

refactor: add quantizer

e82d73f

refactor: update meta

829f53e

refactor: update meta

f084630

refactor: add fp32 quantizer

c3c4ca3

refactor: add quantizer

ec4ea1f

refactor: quantizer

e36f0c5

refactor: add scalar

c1cdd0b

fix: fix cosine

628982d

fix: add quantizer

de33220

fix: fix quantizer

49780f8

fix: remove unused var

b1d4849

fix: header

f0fdf5d

fix: header

0c4cec6

fix: header

73f5bc3

add preprocessor

89843dd

fix: fix symbol

437a754

add untest

e16cda5

Merge remote-tracking branch 'origin/refactor/turbo_quantizer' into r…

22b87d8

…efactor/turbo_preprocessor

remove extra doc

f3a6782

tmp

81d8eb5

tmp

f9b43f1

tmp

fc7efd8

Merge remote-tracking branch 'upstream/main' into refactor/turbo_prep…

642b8bf

…rocessor # Conflicts: # .gitignore

style: fix clang-format issues from upstream merge

79a08eb

style: fix clang-format issues in turbo module

78ace2b

Merge branch 'refactor/turbo_preprocessor' into refactor/turbo_pq

54c9b05

fix: LNK1120

b80bc4d

Merge remote-tracking branch 'origin/refactor/turbo_preprocessor' int…

71b33df

…o refactor/turbo_pq

Copilot AI reviewed Jul 24, 2026

Copilot AI review requested due to automatic review settings July 24, 2026 08:44

Copilot AI reviewed Jul 24, 2026

fix LNK2005

bedd5c3

zzlin237 force-pushed the refactor/turbo_pq branch from 6a04997 to bedd5c3 Compare July 24, 2026 09:58

Copilot AI review requested due to automatic review settings July 24, 2026 09:58

Copilot AI reviewed Jul 24, 2026

Copilot AI review requested due to automatic review settings July 27, 2026 06:37

Copilot AI reviewed Jul 27, 2026

Copilot AI review requested due to automatic review settings July 27, 2026 06:45

richyreachy force-pushed the refactor/turbo_pq branch from 42bbd1b to bedd5c3 Compare July 27, 2026 06:45

Copilot AI reviewed Jul 27, 2026

zzlin237 added 2 commits July 28, 2026 15:15

add fp16 in pq

952521e

fix format of fp16

9ef03dc

Copilot AI review requested due to automatic review settings July 28, 2026 08:31

Copilot AI reviewed Jul 28, 2026

remote warning

d52a50a

Copilot AI review requested due to automatic review settings July 28, 2026 10:13

Copilot AI reviewed Jul 28, 2026

feihongxu0824 reviewed Jul 30, 2026

View reviewed changes

restruct

8adc58e

Copilot AI review requested due to automatic review settings July 30, 2026 03:42

Copilot AI reviewed Jul 30, 2026

Merge remote-tracking branch 'upstream/main' into refactor/turbo_pq

415df61

# Conflicts: # src/include/zvec/core/interface/index_param.h

Copilot AI review requested due to automatic review settings July 30, 2026 06:07

Copilot AI reviewed Jul 30, 2026

richyreachy reviewed Jul 30, 2026

View reviewed changes

Uh oh!

Conversation

zzlin237 commented Jul 7, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

HNSW

L2 Metric

Cosine Metric

Inner Product Metric

Decentralization

IVF

L2 Metric

Cosine Metric

Inner Product Metric

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

zzlin237 commented Jul 7, 2026 •

edited

Loading