Vector index changes can affect query throughput or recall without failing correctness tests. Reviewers currently need to run benchmarks manually to compare a PR with its base revision.
Add a lightweight CI benchmark that compares both revisions on the same runner with the same workload and benchmark driver. The report should make recall and query-throughput changes easy to scan, retain raw samples for investigation, and use advisory thresholds rather than fail CI on noisy timing differences.
The initial scope is a small, repeatable warm-cache workload covering IVF-FLAT, IVF-SQ, IVF-PQ, IVF-RQ and DiskANN. Results should be available in the Actions summary, with an updated PR comment where token permissions allow it.
Implementation: #95.
Vector index changes can affect query throughput or recall without failing correctness tests. Reviewers currently need to run benchmarks manually to compare a PR with its base revision.
Add a lightweight CI benchmark that compares both revisions on the same runner with the same workload and benchmark driver. The report should make recall and query-throughput changes easy to scan, retain raw samples for investigation, and use advisory thresholds rather than fail CI on noisy timing differences.
The initial scope is a small, repeatable warm-cache workload covering IVF-FLAT, IVF-SQ, IVF-PQ, IVF-RQ and DiskANN. Results should be available in the Actions summary, with an updated PR comment where token permissions allow it.
Implementation: #95.