Skip to content

feat(dapi): composite document queries on the getDocuments V1 wire - #4599

Merged
QuantumExplorer merged 1 commit into
feat/composite-document-queries-drivefrom
feat/composite-document-queries-wire
Sep 5, 2026
Merged

feat(dapi): composite document queries on the getDocuments V1 wire#4599
QuantumExplorer merged 1 commit into
feat/composite-document-queries-drivefrom
feat/composite-document-queries-wire

Conversation

@QuantumExplorer

Copy link
Copy Markdown
Member

Issue being fixed or feature implemented

Second PR of the composite-query stack (core in #4598): puts composite document queries on the typed getDocuments V1 wire and serves them from drive-abci. A composite query is a page plus sub-queries derived from its proven results (joins, lookups, counts, siblings), answered as one merged proof; the motivating case is a social feed whose per-page enrichment is today a burst of dependent DAPI calls.

What was done?

Wire (platform.proto)

  • GetDocumentsRequestV1.sub_queries (field 14, repeated SubQuery): each sub-query carries its own contract (empty = the page's), document type, fixed where_clauses / order_by, optional limit, a Kind (DOCUMENTS | COUNT) and an optional Binding { source, source_property, field } whose source is 0 for the page or n for sub_queries[n-1]. Presence selects composite mode; the request's own clauses describe the page. chained and sub_queries are mutually exclusive. No CBOR anywhere: typed clauses only, like the rest of V1.
  • ResultData.composite (variant 7, CompositeDocuments): the page documents plus one SubQueryResult per sub-query (Documents or CountEntries) on the no-proof path. The proof path is the single merged proof in the standard envelope.
  • build.rs: serde default on sub_queries so mock vectors captured before the field stay wire-compatible (same rule as chained).

drive-abci (dispatch/composite.rs)

Routes before the SELECT machinery. Gates: page limit required in [1, max_query_limit]; selects empty or a single DOCUMENTS projection; group_by, having, cursors, offset and time-range clauses (page or sub-query) refused; each distinct sub-query contract fetched once; kinds, limits and bindings decoded into drive's typed shapes; drive's validate failures surface as query errors. Proof path: query_composite_documents_with_proof under the standard proof envelope. No-proof path: documents serialized with their own type and contract, counts as CountEntry rows.

An old node ignores field 14 (proto3 unknown field) and serves the plain page; the verifier in #4598 refuses that page-only proof whenever a sub-query derived anything.

Mechanical: every existing GetDocumentsRequestV1 literal gains sub_queries: Vec::new(); the proof verifier's result-variant name table gains the composite arm.

How Has This Been Tested?

dispatch/composite.rs tests against the yappr-feed fixture plus the dashpay contract:

  • should_return_the_page_and_every_sub_result_without_proof: like counts, quoted-post join, cross-contract profile lookup (a missing profile is a proven absence)
  • should_prove_end_to_end_through_the_v1_wire: the wire proof verifies through DriveCompositeDocumentQuery::verify_composite_documents_proof rebuilt client-side
  • should_require_an_explicit_page_limit, should_reject_chained_and_composite_together, should_reject_an_unknown_sub_query_contract, should_surface_shape_rejections_as_query_errors

The whole document_query::v1 module (103 tests, chained included) passes; clippy --all-targets -D warnings clean on drive-abci, dapi-grpc, drive-proof-verifier and dash-platform-queries; dash-sdk and wasm-sdk compile against the new proto.

Regenerated JS gRPC clients follow in a separate commit on this branch (Docker regen).

Breaking Changes

None. Additive wire fields; old clients and old nodes are unaffected (an old node answering a composite request fails closed on the client).

Checklist

  • I have performed a self-review of my own code
  • I have commented my code, particularly in hard-to-understand areas
  • I have added or updated relevant unit/integration/functional/e2e tests
  • I have added "!" to the title and described breaking changes in the corresponding section if my code contains any
  • I have made corresponding changes to the documentation if needed

🤖 Generated with Claude Code

@coderabbitai

coderabbitai Bot commented Sep 4, 2026

Copy link
Copy Markdown
Contributor

Important

Review skipped

Auto reviews are disabled on base/target branches other than the default branch.

Please check the settings in the CodeRabbit UI or the .coderabbit.yaml file in this repository. To trigger a single review, invoke the @coderabbitai review command.

⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Team

Run ID: df283f2f-42ee-4b3d-a78a-b739f9089ac0

You can disable this status message by setting the reviews.review_status to false in the CodeRabbit configuration file.

Use the checkbox below for a quick retry:

  • 🔍 Trigger review

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@thepastaclaw

thepastaclaw commented Sep 4, 2026

Copy link
Copy Markdown
Collaborator

🕓 Ready for review — 55 ahead in queue (commit 5d3b151)
Queue position: 56/59 · 2 reviews active
ETA: start ~2026-09-06 10:13 UTC · complete ~2026-09-06 11:08 UTC (median 54m across 30 recent reviews; 2 slots)
Queued 5m ago · Last checked: 2026-09-05 09:30 UTC

`GetDocumentsRequestV1.sub_queries` (field 14): each entry is a typed
sub-query (contract, document type, fixed where/order clauses, limit,
DOCUMENTS or COUNT) with an optional binding `<field> IN <values>` whose
values the node derives from the page's, or an earlier sub-query's,
proven documents. Presence selects composite mode: the request's own
clauses describe the page; `chained` and `sub_queries` are mutually
exclusive. `ResultData.composite` (variant 7) carries the page and one
result per sub-query (documents, or count entries keyed by the bound
value) for the no-proof path; the proof path is the single merged proof
in the standard envelope, verified client-side by re-deriving the whole
composition from the proven page.

drive-abci intercepts composite requests before select routing
(`dispatch/composite.rs`): the page limit is required (1..=max), every
SQL-shaped knob, cursor and time-range clause is refused, each distinct
sub-query contract is fetched once, and drive's shape validation
surfaces as client-attributable query errors. Old nodes ignore the
field and serve a page-only proof, which the verifier refuses.

Existing V1 request literals gain the new field; the serde default
keeps mock vectors captured before it wire-compatible.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
@QuantumExplorer
QuantumExplorer force-pushed the feat/composite-document-queries-drive branch from ec02c48 to 34a30e4 Compare September 5, 2026 09:23
@QuantumExplorer
QuantumExplorer force-pushed the feat/composite-document-queries-wire branch from 70e5ef4 to 5d3b151 Compare September 5, 2026 09:23
@QuantumExplorer
QuantumExplorer merged commit 5d3b151 into feat/composite-document-queries-drive Sep 5, 2026
3 of 4 checks passed
@QuantumExplorer

Copy link
Copy Markdown
Member Author

Closed by GitHub as merged when a mistaken force-push briefly collapsed the stack's branches onto one commit; nothing from this PR is in v4.2-dev. Replacement with the same branch and content: #4603.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants