Skip to content

docs: Bring Your Own Bucket onboarding guide#18

Merged
alukach merged 9 commits into
mainfrom
docs/330-federated-s3-onboarding
Jul 20, 2026
Merged

docs: Bring Your Own Bucket onboarding guide#18
alukach merged 9 commits into
mainfrom
docs/330-federated-s3-onboarding

Conversation

@alukach

@alukach alukach commented Jun 20, 2026

Copy link
Copy Markdown
Contributor

Draft. Addresses source-cooperative/source.coop#330 (app-side epic source-cooperative/source.coop#325).

Cross-repo, so it won't auto-close #330 — I'll close that manually on merge.

What

A new Bring Your Own Bucket guide under Using Source (docs/using-source/bring-your-own-bucket.md, added to the sidebar after Access Data Through the Source Data Proxy).

BYOB lets Source serve data from an S3 bucket you own, public or private. The guide is framed around that general concept:

  • Beta / staff-enabled. Leads with the note that BYOB is a beta feature Source staff must turn on (hello@source.coop).
  • Add the backend. Contact Source with the bucket name, region, prefix, and whether it's public or private; Source provisions a connection and returns a connection ID. Public buckets are done at this step.
  • Set up private access (private buckets only). The proxy is an OIDC IdP; it assumes a customer IAM role via AssumeRoleWithWebIdentity at request time — Source stores only the role ARN, no secret at rest.
    • The federation contract table: issuer https://data.source.coop, audience source-coop-data-proxy, subject scv1:conn:<connection-id>:<account>/<product> (trust-policy wildcard scv1:conn:<connection-id>:*).
    • Step-by-step: create the OIDC provider → create the role (trust policy + prefix-scoped read permission policy) → send Source the role ARN.
    • CloudFormation and Terraform, parameterized by connection id, bucket, and prefix, tucked into collapsible <details> blocks.
    • Troubleshooting.

Matches the design pivot: one fixed product-grained subject → a single trust-policy template. Pairs with the in-UI sub-pattern preview in source-cooperative/source.coop#377.

⚠️ Why draft — needs confirmation before publishing

  1. Feature isn't live. The proxy-side federation (data.source.coop #137/#141) isn't shipped; https://data.source.coop/.well-known/openid-configuration currently returns 403. The guide documents an unreleased flow.
  2. Confirm the contract values against the deployed proxy before publishing: the exact aud (source-coop-data-proxy), the issuer host, and the sub format (scv1:conn:{id}:{account}/{product}). These come from the epic design; they must match what the proxy actually mints.
  3. Docusaurus build not run locally (would need a full npm install in the sandbox) — relying on CI/preview to confirm it builds. The doc is plain .md matching existing conventions (<details>/<summary> render natively in MDX), and the sidebar id matches the file path.

🤖 Generated with Claude Code

Adds a "Connect a Private S3 Bucket" guide under Using Source: how Source serves
a private bucket via OIDC federation (Source stores only a role ARN, no
credentials), the federation contract (issuer / audience / subject), step-by-step
IAM setup, and copy-paste CloudFormation + Terraform parameterized by connection
id, bucket, and prefix.

Addresses source-cooperative/source.coop#330.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
@vercel

vercel Bot commented Jun 20, 2026

Copy link
Copy Markdown

The latest updates on your projects. Learn more about Vercel for GitHub.

Project Deployment Actions Updated (UTC)
docs-source-coop Ready Ready Preview, Comment Jul 20, 2026 10:55pm

Request Review

Generalize the guide to cover public or private buckets, lead with the
beta/staff-enablement note, and split into "add the backend" then
"set up private access". Move CloudFormation/Terraform into details tags.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
@alukach alukach changed the title docs: onboarding guide for federated private S3 backends docs: Bring Your Own Bucket onboarding guide Jul 20, 2026
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
alukach and others added 2 commits July 20, 2026 13:06
… Product

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
@alukach
alukach marked this pull request as ready for review July 20, 2026 20:18
@alukach
alukach requested a review from PowerChell July 20, 2026 20:19
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
@alukach
alukach merged commit d05cdc4 into main Jul 20, 2026
2 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants