Skip to content

feat(dapi): composite document queries on the getDocuments V1 wire - #4603

Open
QuantumExplorer wants to merge 1 commit into
feat/composite-document-queries-drivefrom
feat/composite-document-queries-wire
Open

feat(dapi): composite document queries on the getDocuments V1 wire#4603
QuantumExplorer wants to merge 1 commit into
feat/composite-document-queries-drivefrom
feat/composite-document-queries-wire

Conversation

@QuantumExplorer

Copy link
Copy Markdown
Member

Replaces #4599, which GitHub closed as merged when a mistaken force-push briefly collapsed the stack's branches onto one commit. Same branch, same content, same base.

Issue being fixed or feature implemented

Second PR of the composite-query stack (core in #4598): puts composite document queries on the typed getDocuments V1 wire and serves them from drive-abci. A composite query is a page plus sub-queries derived from its proven results (joins, lookups, counts, siblings), answered as one merged proof; the motivating case is a social feed whose per-page enrichment is today a burst of dependent DAPI calls.

What was done?

Wire (platform.proto)

  • GetDocumentsRequestV1.sub_queries (field 14, repeated SubQuery): each sub-query carries its own contract (empty = the page's), document type, fixed where_clauses / order_by, optional limit, a Kind (DOCUMENTS | COUNT) and an optional Binding { source, source_property, field } whose source is 0 for the page or n for sub_queries[n-1]. Presence selects composite mode; the request's own clauses describe the page. chained and sub_queries are mutually exclusive. No CBOR anywhere: typed clauses only, like the rest of V1.
  • ResultData.composite (variant 7, CompositeDocuments): the page documents plus one SubQueryResult per sub-query (Documents or CountEntries) on the no-proof path. The proof path is the single merged proof in the standard envelope.
  • build.rs: serde default on sub_queries so mock vectors captured before the field stay wire-compatible (same rule as chained).

drive-abci (dispatch/composite.rs)

Routes before the SELECT machinery. Gates: page limit required in [1, max_query_limit]; selects empty or a single DOCUMENTS projection; group_by, having, cursors, offset and time-range clauses (page or sub-query) refused; each distinct sub-query contract fetched once; kinds, limits and bindings decoded into drive's typed shapes; drive's validate failures surface as query errors. Proof path: query_composite_documents_with_proof under the standard proof envelope. No-proof path: documents serialized with their own type and contract, counts as CountEntry rows.

An old node ignores field 14 (proto3 unknown field) and serves the plain page; the verifier in #4598 refuses that page-only proof whenever a sub-query derived anything.

Mechanical: every existing GetDocumentsRequestV1 literal gains sub_queries: Vec::new(); the proof verifier's result-variant name table gains the composite arm.

How Has This Been Tested?

dispatch/composite.rs tests against the yappr-feed fixture plus the dashpay contract:

  • should_return_the_page_and_every_sub_result_without_proof: like counts, quoted-post join, cross-contract profile lookup (a missing profile is a proven absence)
  • should_prove_end_to_end_through_the_v1_wire: the wire proof verifies through DriveCompositeDocumentQuery::verify_composite_documents_proof rebuilt client-side
  • should_require_an_explicit_page_limit, should_reject_chained_and_composite_together, should_reject_an_unknown_sub_query_contract, should_surface_shape_rejections_as_query_errors

The whole document_query::v1 module (103 tests, chained included) passes; clippy --all-targets -D warnings clean on drive-abci, dapi-grpc, drive-proof-verifier and dash-platform-queries; dash-sdk and wasm-sdk compile against the new proto.

Regenerated JS gRPC clients follow in a separate commit on this branch (Docker regen).

Breaking Changes

None. Additive wire fields; old clients and old nodes are unaffected (an old node answering a composite request fails closed on the client).

Checklist

  • I have performed a self-review of my own code
  • I have commented my code, particularly in hard-to-understand areas
  • I have added or updated relevant unit/integration/functional/e2e tests
  • I have added "!" to the title and described breaking changes in the corresponding section if my code contains any
  • I have made corresponding changes to the documentation if needed

🤖 Generated with Claude Code

`GetDocumentsRequestV1.sub_queries` (field 14): each entry is a typed
sub-query (contract, document type, fixed where/order clauses, limit,
DOCUMENTS or COUNT) with an optional binding `<field> IN <values>` whose
values the node derives from the page's, or an earlier sub-query's,
proven documents. Presence selects composite mode: the request's own
clauses describe the page; `chained` and `sub_queries` are mutually
exclusive. `ResultData.composite` (variant 7) carries the page and one
result per sub-query (documents, or count entries keyed by the bound
value) for the no-proof path; the proof path is the single merged proof
in the standard envelope, verified client-side by re-deriving the whole
composition from the proven page.

drive-abci intercepts composite requests before select routing
(`dispatch/composite.rs`): the page limit is required (1..=max), every
SQL-shaped knob, cursor and time-range clause is refused, each distinct
sub-query contract is fetched once, and drive's shape validation
surfaces as client-attributable query errors. Old nodes ignore the
field and serve a page-only proof, which the verifier refuses.

Existing V1 request literals gain the new field; the serde default
keeps mock vectors captured before it wire-compatible.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
@thepastaclaw

thepastaclaw commented Sep 5, 2026

Copy link
Copy Markdown
Collaborator

🕓 Ready for review — 55 ahead in queue (commit 3fed9ed)
Queue position: 56/58 · 2 reviews active
ETA: start ~2026-09-07 14:43 UTC · complete ~2026-09-07 15:38 UTC (median 54m across 30 recent reviews; 2 slots)
Queued 1d 4h ago · Last checked: 2026-09-06 14:00 UTC

@coderabbitai

coderabbitai Bot commented Sep 5, 2026

Copy link
Copy Markdown
Contributor

Important

Review skipped

Auto reviews are disabled on base/target branches other than the default branch.

Please check the settings in the CodeRabbit UI or the .coderabbit.yaml file in this repository. To trigger a single review, invoke the @coderabbitai review command.

⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Team

Run ID: 0ac222f6-1201-4aa2-bddd-bea498a8af90

You can disable this status message by setting the reviews.review_status to false in the CodeRabbit configuration file.

Use the checkbox below for a quick retry:

  • 🔍 Trigger review

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants