refactor(mirrors): drop denormalized ProductMirror.storage_type - #456
Draft
alukach wants to merge 7 commits into
Draft
refactor(mirrors): drop denormalized ProductMirror.storage_type#456alukach wants to merge 7 commits into
alukach wants to merge 7 commits into
Conversation
Creating a GCP data connection persisted `details.provider: "gcp"`, but the
data proxy reads `details.provider` directly and only accepts "gcs"/"gs"
for Google (never "gcp"); the same latent mismatch existed for Azure ("az"
vs the accepted "azure"). The provider→storage_type map already emitted the
right values, so mirrors were fine — only the stored provider string was
wrong.
Rename the DataProvider values to what the backend accepts: Azure "az"→
"azure", GCP→GCS "gcp"→"gcs". Member GCP→GCS, schema/type Gcp*→Gcs*. Auth
type stays `gcp_workload_identity` (WIF is a GCP-platform feature, not
storage). Provider labels spelled out in the form.
Legacy rows: reads bypass Zod, so `normalizeConnection` in the
data-connections DynamoDB client remaps "az"→"azure" / "gcp"→"gcs" on
fetchById/listAll. Marked ponytail; drop after backfill.
The now-redundant provider→storage_type translation is left in place here
and removed in the follow-up that drops ProductMirror.storage_type.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Nothing branched on `storage_type`: the data proxy keys off the connection's `details.provider`, and the only source.coop consumer was a display label. With provider values now equal to the backend vocabulary (previous commit), the provider→storage_type maps in createProduct / addProductMirror are pure identity — so remove the field and both maps. - Drop `storage_type` from ProductMirrorSchema; delete both STORAGE_TYPE_BY_PROVIDER maps (mirror no longer stores it). - Mirror "Type" label now reads the provider from connectionInfo (sourced from the connection's details.provider on the edit page). - No migration: ProductMirrorSchema is a plain z.object, so existing records still carrying storage_type parse fine (unknown key stripped). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
This was referenced Jul 22, 2026
|
The latest updates on your projects. Learn more about Vercel for GitHub.
|
Contributor
|
Claude finished @alukach's task in 1m 7s —— View job Review of PR #456
✅ No blocking issues — safe to merge.
Note: I was unable to run |
The azure-data-connection fixture still carried the pre-rename provider value; AzureDataConnectionSchema now requires "azure", and utils.mock.ts eagerly parses this fixture at module load, so every test importing it (and local dev seeding via init-local.ts) threw. Caught in CI review. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Replace the runtime normalizeConnection read-shim (LEGACY_PROVIDER_ALIASES) with a one-time migration script that converts persisted provider values "az"->"azure" / "gcp"->"gcs". Keeps the DB read path free of legacy-value handling; the conversion is done once at deploy instead of on every read. - Remove normalizeConnection + its test; fetchById/listAll return rows as-is. - Add scripts/migrate-provider-values.ts, matching the existing migration convention (positional table arg, DynamoDB scan + guarded UpdateCommand, progress logging). Dry-run by default; --apply to write. Re-runnable (canonical rows are skipped). Run the migration (dry-run, then --apply) before deploying the value rename. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
# Conflicts: # src/lib/clients/database/data-connections.ts # src/types/data-connection.ts
alukach
added a commit
that referenced
this pull request
Jul 22, 2026
#455) ## Problem Creating a GCP data connection persisted `details.provider: "gcp"`, but the data proxy (data.source.coop) reads `details.provider` directly and only accepts `"gcs"`/`"gs"` for Google — never `"gcp"`. The same latent mismatch existed for Azure (`"az"` vs the accepted `"azure"`). The `provider→storage_type` map already emitted the right values, so mirrors were fine — **only the stored provider string was wrong**. ## Change — rename `DataProvider` values to what the backend accepts ``` enum DataProvider { S3 = "s3", Azure = "azure" /* was "az" */, GCS = "gcs" /* was GCP="gcp" */ } ``` - Member `GCP`→`GCS`, schema/type `Gcp*`→`Gcs*`. Auth type stays `gcp_workload_identity` (WIF is a GCP-*platform* feature, not storage). Provider labels spelled out in the form. - **Maps kept** — the now-redundant `provider→storage_type` translation is left in place here and removed in the stacked follow-up (#456). ## Migration (legacy `az`/`gcp` rows) A one-time script — `scripts/migrate-provider-values.ts` — converts persisted `details.provider` values (`"az"→"azure"`, `"gcp"→"gcs"`), matching the repo's existing migration convention (positional table arg, DynamoDB scan + guarded `UpdateCommand`, progress logging). **Dry-run by default; `--apply` to write.** Re-runnable (canonical rows are skipped). Run it (dry-run, then `--apply`) at/before deploy: ``` npx tsx scripts/migrate-provider-values.ts sc-prod-data-connections # preview npx tsx scripts/migrate-provider-values.ts sc-prod-data-connections --apply # write ``` (Earlier revisions used a runtime read-shim; replaced with this script so the DB read path carries no legacy-value handling.) ## Backend compatibility (verified against data.source.coop) **S3** unchanged. **Azure** accepts both `"az"` and `"azure"` → compatible during rollout. **GCS**: old `"gcp"` never worked (hit the error arm); `"gcs"` is accepted. Pairs with data.source.coop#191 (enables the GCS backend). ## Verification `tsc` clean · `jest` affected suites green · lint clean · fixture `provider` fixed (`az`→`azure`) so module-load parsing passes. 🤖 Generated with [Claude Code](https://claude.com/claude-code) --------- Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
…-type # Conflicts: # src/lib/actions/product-mirrors.test.ts # src/lib/actions/product-mirrors.ts # src/lib/actions/products.ts # src/types/data-connection.ts
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Stacked on #455 — review/merge that first; GitHub retargets this to
mainonce it lands.Why
Nothing branched on
storage_type: the data proxy keys off the connection'sdetails.provider, and the only source.coop consumer was a display label. With provider values now equal to the backend vocabulary (#455), theprovider→storage_typemaps are pure identity.Changes
storage_typefromProductMirrorSchema; delete bothSTORAGE_TYPE_BY_PROVIDERmaps.connectionInfo(sourced from the connection'sdetails.provider).ProductMirrorSchemais a plainz.object, so existing records still carryingstorage_typeparse fine (unknown key stripped).Verification
tscclean ·jestaffected suites 116 · lint clean.🤖 Generated with Claude Code