Upload settings_schema.json before validator-consumer assets#7481
Upload settings_schema.json before validator-consumer assets#7481
Conversation
The CLI uploaded `config/settings_schema.json` and `config/settings_data.json` together as the LAST batch of theme files. Server-side validators on blocks, sections, section-group JSON, and template JSON resolve dynamic-source defaults of the form `{{ settings.<theme_setting>.* }}` against the theme's currently-stored schema. On the first push to a fresh dev theme that schema is empty, so any asset whose schema references a not-yet-declared theme setting fails validation — the user sees errors like "Invalid schema: setting with id=... default must be a color or dynamic source access path" or "Dynamic source default value added to '...' is invalid". A second push then succeeds because the user's real schema is now on the server.
Split the `configFiles` partition bucket into `configSchemaFile` and `configDataFile` and reorder the dependent upload chain so schema lands first and data lands last. Schema has no upstream dependencies; data legitimately needs to be last because its current and presets validate against the freshly-uploaded schema. The matching delete order is inverted accordingly.
Closes shop/issues-merchant-workflows#1929
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
graygilmore
left a comment
There was a problem hiding this comment.
I think this makes sense. Spinning up a chat in Slack to talk about some historical bits.
| for (const key of validatorConsumers) { | ||
| const index = flat.indexOf(key) | ||
| expect(index, `${key} missing from dependent chain`).not.toBe(-1) | ||
| expect(index, `${key} should be after settings_schema.json`).toBeGreaterThan(0) | ||
| expect(index, `${key} should be before settings_data.json`).toBeLessThan(flat.length - 1) | ||
| } |
There was a problem hiding this comment.
I have no idea what this test is doing. Is there another way we could show this?
There was a problem hiding this comment.
I removed it. I had it originally to validate the order of files uploaded, but the existing test this PR updates already did that. I wanted to have an extra test enforcing the order since it's crucial, but when I wrote a 'simpler' one, I realized it wasn't adding additional value.
801c97c to
2591058
Compare
2591058 to
bfe4940
Compare
| ...fileSets.otherLiquidFiles, | ||
| ...fileSets.configFiles, | ||
| ...fileSets.configDataFile, | ||
| ...fileSets.configSchemaFile, |
There was a problem hiding this comment.
Does this ordering matter at runtime for deletion as well? Currently it looks like deletion happens in async batches where the ordering might not be guaranteed.
There was a problem hiding this comment.
Effectively it doesn't. My understanding is that mirrors the upload just for consistency, but not an actual contract. So I'll update my inline comment about "config file consideration" just to not make it seem important
But as to whether it actually matters: the utility of deleting is getting the remote theme in sync with local dev. Doesn't matter where settings_schema is because there's no use-case for ever deleting that file and keeping a functioning theme.
| independentFiles: [fileSets.otherLiquidFiles, fileSets.otherJsonFiles, fileSets.staticAssetFiles], | ||
| // Follow order of dependencies: | ||
| dependentFiles: [ | ||
| fileSets.configSchemaFile, |
There was a problem hiding this comment.
If schema upload fails, would you want later dependent uploads to proceed? Current batch upload doesn't throw when bulk upload fails.
There was a problem hiding this comment.
This is a great point, but I don't want to tackle that in this PR because its a larger change to the failure mode of theme push. Today theme uploads behave as "upload as much as possible" and we get reports of what couldn't be uploaded. Adding a dependency check sounds useful, it's a "fail fast" approach - something that we could be benefitting from already given the dependencies for Layout files > JSON templates, Liquid sections > JSON sections, etc. But I'm hesitant to change it just yet as I'm no sure how devs may be benefitting from the current "upload as much as possible" mechanic.
WHY are these changes introduced?
Closes shop/issues-merchant-workflows#1929
This is the CLI counterpart to the new
color_palettesetting type. Thecolor_paletteis declared at the theme level (config/settings_schema.json). Blocks and sections set their own colordefaultvalue via{{ settings.<palette>.<role> }}.Issue: The
shopify theme pushfails on the first push to a fresh dev theme whenever a block/section uses that pattern. The second push to the same theme succeeds, because by thenconfig/settings_schema.jsonhas been uploaded and the server-side validator that runs on the block file can resolve the reference against the stored palette.Example of first push errors:
The second push to the same theme succeeds, because by then the user's
config/settings_schema.jsonhas been uploaded and the server-side validators can resolve the reference.Root cause. The CLI uploads files in dependency order. Today
config/settings_schema.jsonis bundled withconfig/settings_data.jsoninto a singleconfigFilesbucket and uploaded last — after every block/section/section-group/template asset. When the server-side validator on a block file checks a{{ settings.<palette>.<role> }}default, it queries the theme's currently-storedsettings_schema.json, which on a fresh theme is the'[]'placeholder seeded byensureThemeCreation. The reference can't resolve until the user's real schema lands.WHAT is this pull request doing?
Splits the
configFilespartition bucket intoconfigSchemaFileandconfigDataFile, then reorders thedependentFilesupload chain:settings_schema.jsonis purely declarative and has no upstream dependencies, so it's safe to upload first.settings_data.jsonlegitimately needs to be last — itscurrentandpresetsvalidate against the freshly-uploaded schema and can reference sections / templates uploaded earlier. The matchingorderFilesToBeDeletedis updated to delete data before schema (the inverse of the new upload order).Files changed (all in
packages/theme/src/cli/utilities/):theme-fs.ts— splitconfigRegexintoconfigSchemaRegex/configDataRegex; splitpartitionThemeFilesbucket.theme-uploader.ts— reorderdependentFiles(schema head, data tail), invertorderFilesToBeDeleted, expand the header comment explaining the contract. ExportsorderFilesToBeUploadedandChecksumWithSizeso the new test can exercise the function directly.theme-fs.test.ts— partition tests updated for the two new buckets.theme-uploader.test.ts— existing upload-order test updated for the new sequence; new direct unit test oforderFilesToBeUploadedasserts schema-first / data-last and that every validator-consumer asset (blocks, section liquid, section group JSON, template JSON, contextualized template JSON) is sequenced between them.Verified scope. Block-level
color_palettedynamic-source defaults — Horizon'sinput_border_colorrepro succeeds on first push. The same upload-order fix is expected to cover the section-group and template JSON datasource walkers (same dependency on the stored schema), though those weren't independently end-to-end verified.Out of scope. Validator paths that don't consult the stored
settings_schema.jsonat all — for example, thestring_or_datasourcevalidator that runs on URL / link block settings — fail before any schema lookup and aren't affected by reordering. Those need a separate server-side change.How to test your changes?
Test a
color_palettetype input on a store with the new feature enabled.blocks/test.liquidconfig/settings_schema.jsonwith this setting.{ "type": "color_palette", "id": "color", "default": { "background": "#ffffff", "link_color": "#000000" } },pnpm shopify theme push --path=<YOUR_PATH>and verify no error messagesPost-release steps
None.
Checklist
@shopify/theme, changeset added (.changeset/fix-theme-push-fresh-dynamic-source-defaults.md).