Skip to content

flowctl: fetch storage mappings and data planes via GraphQL - #3238

Draft
GregorShear wants to merge 1 commit into
masterfrom
greg/flowctl-gql-storage-mappings
Draft

flowctl: fetch storage mappings and data planes via GraphQL#3238
GregorShear wants to merge 1 commit into
masterfrom
greg/flowctl-gql-storage-mappings

Conversation

@GregorShear

@GregorShear GregorShear commented Jul 23, 2026

Copy link
Copy Markdown
Contributor

Summary

Resolves storage mappings and accessible data planes during flowctl source loading through the control-plane GraphQL API instead of PostgREST.

Why

The PostgREST path relied on row-level security, whose grant graph only walks downward: a user scoped to a sub-prefix (for example, acmeCo/team-a/) could not read the tenant-root mapping at acmeCo/, so flowctl discover failed with an access error. The GraphQL resolvers authorize against the snapshot grant graph, which walks ancestors as well as descendants.

A storage mapping identifies eligible data planes by name, while existing live specs reference their assigned data plane by ID. The dataPlanes GraphQL query now returns both values, allowing local validation to resolve either form without depending on legacy RLS.

How

  • Storage mapping prefixes are fetched with filter: { catalogPrefix: { in: [...] } } exact-set queries, batched at the filter's 100-entry limit.
  • All accessible data planes are fetched through the paginated dataPlanes connection. This is necessary because storage mappings reference planes by name rather than ID.
  • Unreadable resources are filtered by the server. Transport, HTTP, and GraphQL execution errors propagate.
  • Recovery mappings continue to be synthesized client-side with a zero control ID.

The storage mapping filter landed in #3242. Data-plane IDs landed in #3240. Live specs and inferred schemas remain on their existing PostgREST paths.

Testing

@GregorShear
GregorShear force-pushed the greg/flowctl-gql-storage-mappings branch from c02cdef to 31ad5e7 Compare July 23, 2026 20:25
@GregorShear
GregorShear changed the base branch from master to greg/gql-storage-mappings-filter July 23, 2026 20:26
Base automatically changed from greg/gql-storage-mappings-filter to master July 27, 2026 19:57
@GregorShear
GregorShear marked this pull request as draft July 30, 2026 16:50
@GregorShear
GregorShear force-pushed the greg/flowctl-gql-storage-mappings branch 2 times, most recently from 60b9dff to bb982b5 Compare August 17, 2026 19:55
@GregorShear
GregorShear changed the base branch from master to agent/graphql-storage-mapping-raw-spec August 17, 2026 19:56
@GregorShear
GregorShear force-pushed the greg/flowctl-gql-storage-mappings branch from bb982b5 to 3c60052 Compare August 17, 2026 20:37
Base automatically changed from agent/graphql-storage-mapping-raw-spec to master August 17, 2026 21:19
@GregorShear
GregorShear force-pushed the greg/flowctl-gql-storage-mappings branch 3 times, most recently from a196e43 to 79731da Compare August 17, 2026 23:06
Resolve storage mappings and accessible data planes during flowctl source loading through the control-plane GraphQL API. Query relevant storage prefixes in batches of 100, paginate data planes, propagate request failures, and synthesize the recovery mappings required by validation.
@GregorShear
GregorShear force-pushed the greg/flowctl-gql-storage-mappings branch from 79731da to 3c36ba1 Compare August 19, 2026 18:49
@GregorShear GregorShear changed the title flowctl: fetch storage mappings via GraphQL instead of PostgREST flowctl: fetch storage mappings and data planes via GraphQL Aug 19, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant