Qdrant 1.19 added a slice condition to the filter language, for sliced scroll and deterministic
sampling. Kdrant has neither, and the workaround people use instead is the one that goes wrong.
Sampling a collection today means searchMatrixPairs's sample, which samples for the distance matrix
and not for the caller, or a scroll that reads everything and throws most of it away. Splitting a scroll
across N workers means partitioning by point id and hoping the ids are evenly distributed, which they
are not when they are UUIDs from an upstream system.
A slice condition answers both. slice(0, 4) reads a quarter of the collection, four of them read all
of it exactly once, and the split is the server's rather than a guess about the id space. It is
deterministic, so a second run over the same slice reads the same points, which is what makes it usable
for evaluation sets and for reproducible sampling in a test.
This lands in the filter DSL beside the other conditions and in the scroll path, and the interesting
part is not the wire format. It is what the ordered scroll does with it: scroll follows a cursor and
drops the points a page repeats at the boundary, and a sliced scroll has to keep that property per
slice rather than across the whole collection.
Exit criterion: four sliced scrolls over one collection read every point exactly once between them,
asserted in the shared contract against a real Qdrant, and a second pass over one slice reads the same
points as the first.
Qdrant 1.19 added a slice condition to the filter language, for sliced scroll and deterministic
sampling. Kdrant has neither, and the workaround people use instead is the one that goes wrong.
Sampling a collection today means
searchMatrixPairs'ssample, which samples for the distance matrixand not for the caller, or a scroll that reads everything and throws most of it away. Splitting a scroll
across N workers means partitioning by point id and hoping the ids are evenly distributed, which they
are not when they are UUIDs from an upstream system.
A slice condition answers both.
slice(0, 4)reads a quarter of the collection, four of them read allof it exactly once, and the split is the server's rather than a guess about the id space. It is
deterministic, so a second run over the same slice reads the same points, which is what makes it usable
for evaluation sets and for reproducible sampling in a test.
This lands in the filter DSL beside the other conditions and in the scroll path, and the interesting
part is not the wire format. It is what the ordered scroll does with it:
scrollfollows a cursor anddrops the points a page repeats at the boundary, and a sliced scroll has to keep that property per
slice rather than across the whole collection.
Exit criterion: four sliced scrolls over one collection read every point exactly once between them,
asserted in the shared contract against a real Qdrant, and a second pass over one slice reads the same
points as the first.