Skip to content

M68 · The comparison benchmark has a harness and no numbers #121

Description

@tonytonycoder11

M51's exit criterion says: a published benchmark reports latency and throughput for Kdrant and the
official Java client across the four named operations, with the harness in the repository, and the
write-up names where Kdrant loses or states that nothing was found.

2.2.0 shipped the harness. It did not publish the numbers, and the board reads Shipped against an
item whose criterion is half met.

That gap matters more than an unfinished task usually would, because of what the missing half is. The
whole argument for this client is ergonomics: suspending functions, a typed filter DSL, no boxing on the
hot path. A reader who is unconvinced assumes the ergonomics were bought with throughput, and nothing in
the repository contradicts them. The harness exists precisely to contradict them and has not been run.

The work is a workflow dispatch and a table, not a project. OfficialClientComparisonBenchmark runs on
the Benchmarks workflow against a clean runner with Qdrant's gRPC port exposed, which is what the M51
work already arranged. The result goes into benchmarks/README.md with the run id, the way the existing
latency table does.

The part that needs judgement rather than a command is the write-up. If Kdrant loses a row, the row is
published and the reason is worked out rather than omitted, and REST against gRPC is the first thing to
check before concluding anything about the client. If it wins every row, the write-up says what was
tried and why the reader should believe a set of rows the author chose.

Until it exists, the comparison table in the README stays a footprint comparison and nothing in this
repository claims anything about speed against the official client.

Exit criterion: benchmarks/README.md carries latency and throughput for both clients across single
search, batch search, large-batch upsert and full scroll, naming the run they came from, and states
either where Kdrant is slower or what was tried and found nothing.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    • Status
      Shipped

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions