Skip to content

v1 getDocuments: tracker for wire-only / not_yet_implemented surfaces from #3654 #3655

Description

@QuantumExplorer

Status

  • (2) SUM and (3) AVG: ✅ landed in feat(drive): document sum + average proof primitives, with SDK fan-out scaffolding and reproducible benchmarks #3661 (drive sum/avg dispatchers wired end-to-end on no-proof and proof paths; SDK DocumentSum / DocumentAverage + DocumentSplitSums / DocumentSplitAverages FromProof impls; ABCI routing through SelectFunction::Sum / ::Avg; wasm-sdk bindings getDocumentsSum / getDocumentsAverage + their WithProofInfo variants). The polymorphic AggregateValue response variant didn't end up needed — SUM and AVG ride dedicated data.sums / data.averages response slots alongside data.counts.
  • Remaining items (1, 4–14) still wire-only.

Background

PR #3654 lands the v1 getDocuments SQL-shaped wire surface (SELECT / WHERE / GROUP BY / HAVING / ORDER BY / LIMIT / OFFSET), but ships several pieces as wire-only: the proto carries them so callers can encode the shape and the future server capability can land without another wire-format change, but the server rejects them at routing time with QuerySyntaxError::Unsupported("… is not yet implemented").

This tracker collects every not_yet_implemented introduced by that PR so each can be picked up as a focused follow-up.

Inventory of not_yet_implemented rejections (rs-drive-abci v1 handler)

# Reject site Notes
1 HAVING clause is not yet implemented Non-empty having rejected wholesale. Decoded Vec<HavingClause> already lands in having_clauses_from_proto (currently #[allow(dead_code)]); execution is "flip the gate + thread into the dispatchers."
2 SELECT SUM is not yet implementedDone in #3661. Drive drive_document_sum_query dispatcher routes (mode × range × group_by × prove) to dedicated executors (point_lookup, total, range_no_proof, range_proof, range_distinct_proof, range_aggregate_carrier_proof, per_in_value). Response shape: data.sums.aggregate_sum (single) / data.sums.entries (per-group), proven via grovedb's verify_aggregate_sum_query.
3 SELECT AVG is not yet implementedDone in #3661. AVG composes count + sum walks under the hood (no separate grovedb primitive); the dispatcher resolves each AVG shape to its matching SUM mode and the client folds sum / count. Response shape: data.averages.aggregate_average / data.averages.entries. Single-aggregate fold uses checked_add on both u64 (count) and i64 (sum) axes; per-group flat-map fold likewise.
4 SELECT MIN is not yet implemented Per-group/global MIN executor needed (semantically distinct from HavingRanking::Min which is cross-group). Waits on a grovedb min/max primitive — the order-by-then-LIMIT-1 emulation has the wrong proof shape for the cryptographic verifier.
5 SELECT MAX is not yet implemented Symmetric to MIN. Same grovedb-primitive dependency.
6 SELECT COUNT(field) is not yet implemented Counting non-null values of a specific field; today only COUNT(*) (empty field) is evaluated. Needs a non-null counter walk that doesn't exist in grovedb.
7 GROUP BY with SELECT DOCUMENTS is not yet implemented Documents-fetch path doesn't accept grouping.
8 GROUP BY on field '<x>' which is not constrained by an In or range where clause is not yet implemented Single-field GROUP BY on a field not present in WHERE.
9 two-field GROUP BY outside the (In, range) compound shape is not yet implemented Only the canonical compound (in_field, range_field) ordering is wired; other orderings need a new merk walk.
10 GROUP BY with more than two fields is not yet implemented group_by.len() > 2. Probably permanent in current architecture; flag for confirmation.
11 OFFSET pagination is not yet implemented Wire-reserved optional uint32 offset = 12;. Cursor pagination via start_after / start_at is the supported path today.
12 multi-projection SELECT is not yet implemented Wire: repeated Select selects. Today selects.len() > 1 rejected. Response shape needs a parallel repeated AggregateValue values field when this lands.
13 start_after / start_at with SELECT COUNT is not yet implemented Paginating count results — workaround today is narrowing the range clause.
14 ORDER BY on aggregate keys is not yet implemented Wire: OrderClause.target.aggregate. Drive's OrderClause only carries a plain field: String; extension needs a new OrderTarget enum at the drive level. Overlaps HavingRanking::Top/Bottom for top-N selection but more general.

Dependencies between items

  • (2) (3) (4) (5) all need a shared numeric-aggregate executor in drive + a polymorphic AggregateValue response variant. (2) and (3) landed in feat(drive): document sum + average proof primitives, with SDK fan-out scaffolding and reproducible benchmarks #3661 via dedicated data.sums / data.averages response slots — the polymorphic AggregateValue variant turned out unnecessary for the SUM/AVG-only case. (4) MIN and (5) MAX still wait on a grovedb min/max primitive (separate from the SUM/AVG plumbing).
  • (12) depends on (4) / (5) for the MIN/MAX projections to be combinable; SUM and AVG could already participate in a multi-projection response if the response shape were extended. Practical implementation likely waits for MIN/MAX so the full surface lands at once.
  • (14) depends on extending drive's OrderClause shape; once done, HavingRanking::Top/Bottom could potentially be consolidated through this path. The SUM/AVG ranking infrastructure (per-group entries with (in_key, key, sum) / (in_key, key, count, sum) triples) is now in place; ordering on those metrics is the remaining piece.
  • (1) is independent — the wire decoder is already in conversions.rs::having_clauses_from_proto; only the gate-flip + dispatcher threading is needed. Now that SUM/AVG executors exist, HAVING aggregates over them (HAVING SUM(x) > 100) have something to evaluate against.
  • (11) is independent — drive already accepts offset on SizedQuery; routing-layer wire-through is the main work.

Suggested first slices

  1. (11) OFFSET — smallest contained piece. Probably one PR.
  2. (1) HAVING — decoder already in tree; executor design is the main work. With SUM/AVG landed, HAVING aggregates over them have concrete evaluation targets. One PR.
  3. (4) (5) MIN / MAX — waits on a grovedb primitive (or a documented decision that the order-by + LIMIT 1 emulation isn't usable for proofs). Out-of-band design work before code.
  4. (14) ORDER BY aggregate — could land independently of MIN/MAX now that SUM/AVG per-group entries exist; reuses the ranking shape already in DocumentSplitSums / DocumentSplitAverages. One PR.
  5. (12) Multi-projection SELECT — depends on (4)/(5) to be useful. One PR once those land.
  6. (7) (8) (9) (10) (13) — GROUP BY / pagination-on-count gaps. Individual PRs as needed.

References

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions