Repository navigation
feat(cf-v2): unified target-driven delivery (Native Dev, smoke modes, migration stage) - #89
Merged
Merged
Conversation
…pe plan) Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
…IP workflow follows) Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
…tion/Deploy/Smoke/Rollback, Native Dev, smoke modes) Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
… (PR #89 review P1-A/P1-B/P2-A/P2-B) - _cf-worker-delivery.yml: one internal reusable workflow (preflight/migrate/deliver) called once per kind; no matrix, no shared output keys - public outputs select their source by plan flags; phase_state carries each kind's own results - canonical Preview URL resolved before the build (Cloudflare preview_url_suffix), asserted after publish; build-env origin checks - Preflight of every enabled kind precedes any migration; shared D1 migrated once by its owner; engine binds the plan's D1 id - contract/job/unit tests rewritten for the new topology plus regression scenarios WIP: test-jobs.py full run and real-runner evidence still pending. Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
…kipped at job level on policy/gate
- rollback job `if` now also requires production policy automatic and a non-blocked cross-service gate (no no-op green rollback job)
- ghexpr: render() respects quoted `}}`, format() unescapes `{{`/`}}`
- regression scenarios for reversed completion order, per-target failure isolation, shared-D1 dedupe, URL pre-resolution, hand-off isolation
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
…penspec tasks; real-runner evidence outstanding Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
… the whole called workflow GitHub validates the ceiling for every job of a called workflow, including jobs skipped by if:. Found by the real-runner scratch run (startup_failure). Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
…ding and hosted-runner pnpm limitation Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
…uts.json with one jq Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
… PR-scoped validation concurrency test.sh fast (~10s) / chain / full; run-job-groups.py runs the 7 test-jobs.py sections as independent processes (379 assertions, same as serial). CI: cancel superseded PR validation only; checks keep the 'contract' job name; delivery workflow added to paths+actionlint. Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
…rget/environment/D1/owner/group copies Slice A. delivery.mjs: versioned descriptor (targetDescriptor/validateDescriptor/buildDescriptors/parseDescriptor); plan emits 'descriptors'; the 12 static calls pass one 'descriptor' instead of 6 fields; delivery workflow and the engine (descriptor-check at init, also supplies smoke mode) consume it. persistent-dev keeps the legacy shared Dev concurrency group (pinned by tests). Plan drops d1_*/owner_of_* outputs. Tests: delivery.test.mjs, contract, test-jobs (4 forged-descriptor refusals). Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
…to delivery.mjs + engine
Slice B. checkPeerPreflights / resolveMigrationUsed (pure, unit-tested incl. every refusal); engine phase
migration-gate runs right after init for the migrate and publish stages, taking owner/D1 from the validated
descriptor. Delivery workflow loses two inline shell steps. A job that fails its gate now reports published=false
in its hand-off (was {}): two isolation assertions compare != 'true'.
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
…mal run and the recovery rebuild Slice C. engine.sh run_build/run_publish replace the duplicated env blocks of build, publish and recover-run; recovery only supplies its state dir, rebound build env/version and no host re-check. Contract test pins one build.sh and one publish.sh invocation. Recovery scenarios (rebuild at last-good SHA, exact URL, re-smoke) unchanged and green. Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
… engine.sh into delivery.mjs Slice D. verificationOutcome (only content-level verifies), prodRollbackBlock (policy, identity, no D1 schema rollback, never over a foreign/newer deployment) and rollbackVerdict (restored_verified needs a content-level re-smoke) are pure and unit-tested for every refusal; the engine gathers facts and performs effects. Shell scenarios unchanged and green. Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
…ries; every group writes its perf report Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
… (P1) phase_state now carries the deploy call's validated descriptor map; the verify plan restores it (restoreDescriptors) and only moves phase deploy -> verify. Cross-checked against kinds, app, repository, D1 ids and migration owner; wrong/forged state fails the plan closed; a legacy hand-off is rebuilt only from the deploy call's own inputs. Unit tests + end-to-end deferred scenarios (skip policy, shared-db lock, tamper cases); mutation check proves the scenarios catch the old bug. Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
…cks) validateInit (delivery.mjs, pure, unit-tested for every refusal) replaces ~6 Node spawns + jq in engine.sh init: target/kind/stage/ mode binding, descriptor, strategy/policy, enabled kind, per-target requirements, URL isolation and env data. Credentials stay an explicit shell check; state files are written only after every check passed. parse-env.mjs entry guard no longer throws when imported. Measured (one Production chain, system-wide execve): node 17 -> 9, total 567 -> 549; engine init 213 ms -> 59 ms avg. Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
…ewriting test-jobs.py is now shared fixtures + eight explicit group_* functions registered in GROUPS (--list / --group ID); the deferred-verification scenarios are their own group. run-job-groups.py is a thin launcher: bounded parallelism (CF_V2_TEST_JOBS or min(4, CPUs)), a MANIFEST of expected groups with minimum check counts, and it fails on a missing/unexpected/empty/shrunken/skipped group or a missing perf report. 393 checks = the 379 before + 14 added this session; no assertion removed or weakened. Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Removes the 'kind' input beside the descriptor from the 12 calls and the delivery workflow (11 uses): one canonical record, no way for a call to disagree with its own descriptor. Contract and harness updated accordingly. Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
…1 ownership, fail-fast Full 1. Deferred verify: the top-level concurrency expression now applies the deploy expression to phase_state.inputs (never the caller-supplied group string); restoreDescriptors requires phase_state.group to equal runLockGroup(deploy inputs) and the Persistent Dev deploy/rollback locks to equal the repository + app + deploy-time d1_lock_scope identity. JS twin (runLockGroup) is contract-tested against the rendered YAML expression on 512 scenarios in deploy and verify mode. 2. cli.mjs workflow annotations (::error/::warning/::notice/::add-mask) go to stderr: stdout is only key=value records, so a rejected input can no longer add an 'Invalid format' error to $GITHUB_OUTPUT. The job harness now enforces GitHub's output-file format. cli.test.mjs. 3. Legacy phase_state: migration ownership is reconstructed per physical D1 only when every member result proves the same owner on the same D1; missing/conflicting evidence fails closed; non-D1 hand-offs unchanged. 4. test.sh full: Chain is not started when Fast failed; tooling pre-check; test-testsh.sh. 404 chain checks (393 + 11), 65 delivery + 7 cli unit tests. Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
…nd four Migration calls - _cf-worker-v2.yml: the eight reusable-workflow calls (preflight_*, migration_*) become ONE job. Its first steps are the read-only pre-mutation checks (stage preflight for each non-owner target: target build, effective-config audit, exact D1 binding), its last engine step migrates once (stage migrate, which validates the owner the same way before the mutation). Environment = the plan's migration owner (<app>/prod on main). Skipped when d1_migrations is off; Delivery proceeds. - plan: single-DB rule. Identical D1s dedupe, distinct physical D1s fail the plan before any mutation; outputs `migration` (owner, environment, d1 id) replaces owns_<kind>. - _cf-worker-delivery.yml: only deploy / smoke / rollback; stage, kinds, peers, own_/owner_migration_* removed, replaced by migration_result / migration_json. A failed Migration skips every dependent Delivery at job level. - engine: init starts from a clean state (several engine calls per job); migration gate only for publish; peer-preflight bookkeeping (checkPeerPreflights) removed. - tests: contract, job-chain, delivery.mjs, cli.mjs updated in the same slice; migration_status is an always-run status step. Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
…Migration documentation and measurements - engine/cf-deploy: the Migration job calls the engine once per validated target; the tooling npm ci and the application Setup run once per job (CFV2_DEPS, reserved env name); init only wipes the engine's own default state dir. - _cf-worker-v2.yml: YAML anchors for the four identical Delivery permissions / secrets blocks and the five identical CI tooling fetches. - docs / OpenSpec: revised decision (no Preflight stage, one Migration job), single-DB compatibility restriction, measured before/after. Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
… and persistent Dev as small reusable workflows - every .cf-tools checkout is gone: shared actions and the delivery/CI/lifecycle/persistent-dev workflows are referenced with $/ (this repository at the running workflow's commit); tools_ref stays accepted but is a deprecated no-op - new _cf-worker-ci.yml (six parallel checks + fail-closed Validation Gate, validation=passed verdict), _cf-worker-lifecycle.yml, _cf-worker-persistent-dev.yml; new actions/cf-tools (TOOLS_DIR + npm ci) for the jobs that run the tooling scripts directly - Plan emits the canonical lock group (runLockGroup); concurrency.group loses its duplicated verify copy; phase_state drops the redundant per-kind d1 map and the duplicated group expression - migration/delivery depend on the plan only (the plan already requires a passed CI) - contract + chain tests ported (529 contract checks, 413 chain checks green) Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
… yet Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
…runner-version note; consumer doc Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
…decision, concurrency note, real-runner evidence @ 3bda5fb Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
resolveMigrationUsed accepted a result with stale=true without checking migration/owner/D1. A Migration job superseded by a newer push succeeds as a skip, so after a force-push back to the original commit (A -> B -> A) the Delivery's branch-tip check saw a fresh tip and published without the required migration. The gate now rejects stale (true or "true") and malformed/missing results; owner and physical D1 stay mandatory. Tests: unit, executed engine gate, chain A->B->A. Docs + OpenSpec updated. Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
…ity audit and rollout blockers Docs and OpenSpec task state only; no implementation, workflow or test change. Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
…re with GitHub's exact message Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
…consumer fix PR status Docs and OpenSpec task state only. Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Refactor Cloudflare Workers v2 CI/CD in place to reduce orchestration duplication while retaining one shared delivery engine, one normal-pipeline D1 Migration job, per-target Smoke/Recovery, deferred verification, and the existing public workflow inputs/outputs.
Current reviewed HEAD:
8ad4d40(includes the P1 stale-migration fix). Latest PR CI: run 41595263650 — passed. This PR has not been merged;main, the floating@v2tag, and consumer repositories have not been changed by this PR.Current DAG
The four Delivery job definitions are conditional; disabled targets do not execute. The normal pipeline has no separate Preflight job/matrix.
stage: preflightremains as read-only validation inside the single Migration job.Key changes
_cf-worker-ci.yml. Same-repo PR does not rerun CI already covered by branch push; fork PRs run credential-free validation.d1_migrations: falseskips Migration without blocking Delivery._cf-worker-delivery.ymlandactions/cf-deployfor Build/Audit/Publish → Smoke → guarded Rollback. Deployment outputs, environment approvals, concurrency and rollback decisions remain target-specific.$/..., binding tooling to the called workflow's exact repository commit and removing the redundant.cf-toolscheckouts.tools_refremains accepted but is deprecated, optional and ignored.phase_statehandoff omits a redundant internald1map; pass it through unchanged. Check-context names do change due to CI extraction.Security / P1 resolution (fixed at HEAD
8ad4d40)The previous
resolveMigrationUsed()accepted a Migration result withstale=truewithout proving that any D1 migration ran. In a potential branch A → B → A force-push race, a later fresh-SHA check would not prove migration completion.Fixed: the publish-stage Migration Gate now rejects stale (
trueor"true"), absent/malformed result, failed/skipped Migration, missing or incorrect owner, non-appliedresult, or mismatched physical D1. The Gate runs before publish-stage build or mutation.d1_migrations=falseand deferredpipeline_phase=verifyremain supported. Regression coverage includes A → B → A, no unauthorized Publish/Smoke/Rollback/Release.Additional protections retained: Production/non-production D1 isolation; effective Wrangler config validation; protected Production Environment; stale-SHA guards; Smoke
full|basic|skip; target-specific recovery; Release only after verified Production; no automatic schema rollback.Scope and metrics
_cf-worker-v2.yml_cf-worker-delivery.ymlThe benefit is separation of responsibilities and less duplicated setup, not lower total YAML line count. Available real-runner comparison (
n=1per scenario) shows no established runtime improvement.Verification evidence
8ad4d40: PR CI run 41595263650 — passed. Fast profile, Chain profile (mocked Cloudflare/GitHub), ShellCheck and Actionlint all passed; includes the stale-migration regression tests.3bda5fb(before P1 fix): PR CI run 41589279882 — passed; cross-repository disposable scratch-caller evidence confirms$/resolution, parallel CI and Validation Gate, D1 gating and skip cases, distinct-D1 rejection, deferred verification, manual Persistent Dev and lifecycle. Examples: parallel CI, CI failure blocks delivery, migration disabled, two D1s rejected, legacy manual Dev, PR lifecycle. Several scratch runs intentionally conclude failure because Cloudflare credentials were withheld; their intended gates and observed steps are recorded in the docs.8ad4d40: done, see Final merge-readiness verification below.Final merge-readiness verification (implementation
8ad4d40, 2026-11-03)Implementation is unchanged since
8ad4d40; a docs-only commit recording the details below (fd6ab85, touchingdocs/cf-worker-v2-refactor.md,docs/cf-worker-v2-unified-delivery.mdand the OpenSpectasks.mdonly) is the PR branch tip; the implementation SHA stays8ad4d40. Nothing was merged;@v2, consumers, branch protection and Cloudflare were not changed.1. Regression at
8ad4d40. PR CI 41595263650 passed. Local re-run on the exact SHA:test.sh fastandtest.sh fullPASSED (561 contract + 420 chain checks), ShellCheck 0.11.0 (+ composite-action scripts) clean,actionlint 1.7.12clean. All nine required assertions have named passing tests (staletrue/"true"blocked; missing/malformed fail closed; valid applied Migration with matching owner + physical D1 succeeds;d1_migrations=falsedoes not block; A → B → A rejected before the build; failed Migration never starts publishing; verify never migrates; Smoke/Rollback target-isolated; Release only after verified Production).2. Real-runner stale gate. cf-v2-scratch-caller run 41598319442 runs the real
cli.mjs migration-gateentrypoint from8ad4d40on the orglinuxrunner andubuntu-latestwith a validated descriptor and synthetic handoffs: stale (trueand"true"), empty, malformed,none, owner mismatch, D1 mismatch and failed-Migration inputs all exit1; valid andd1_migrations=falsecontrols exit0; PATH shims show nowrangler/gh/curl/pnpm/npxcall (no publish-side action). (The first dispatch, 41598162595, failed on a bug in my test script —bash -e— not in the implementation.) The full A → B → A race is not reproducible on Actions without a live account; its evidence stays the chain regression.3. Scratch caller pinned to
8ad4d40(allreferenced_workflowsand nested$/actions resolved to the same SHA; org runners2.338.0; no Cloudflare credentials, sofailureconclusions below are the designed fail-closed result and each was checked against its step message):preview credentials not provided(expected)Database Migration; failed closed on credentials; Deliveries skippednothing was changed4. Consumer compatibility (default branches read 2026-11-03; no other consumer of
_cf-worker-v2.ymlexists; the real planner was run on every call × push/dispatch/PR/delete):45031ada…; branchfeb45b04…)secrets:; update checks193440aa…; devca755ecb…)41fa4337…; dev8e3f8323…)e12e2d9e…; previewb9421085…)311ab3c8…; dev9a69f8e9…)secrets:; update checks2.338.0(nine of ten listed on 2026-11-03;heavenweb-runner-10absent) (≥ 2.336.0, confirmed by API; scratch jobs ran onheavenweb-runner-3). Node/pnpm/Playwright setup is unchanged frommain. Hostedubuntu-latestalso accepted$/but lacks pnpm.403 Upgrade to GitHub Proalthough the token has admin rights (org planfree) — plan limit, not proof that none exist. An administrator must confirm none of these is required, or move it:<job> / Lint|Typecheck|Unit Tests|Integration Tests|Build Validation|E2E[ (variant)]→<job> / CI / …; add<job> / CI / Validation Gate;<job> / Migration→<job> / Database Migration;<job> / Deploy|Smoke Tests|Revert on Verify Failure|Shared Dev*→<job> / Delivery (<kind>) / ….<job>isPipeline(blog, innovation-apps, heaven-www),Heaven ID/Heaven Checkin/Heaven ID Test Client, orcf(anas-mcp).pipeline_phase/phase_state/upstream_gate.5. Live non-production evidence — BLOCKED. The only Cloudflare credentials reachable are the Production org secrets; the scratch repository has none and no disposable, approved account/D1/protected Environment exists. Prerequisites: a scratch Cloudflare account or disposable-scope token, a throw-away D1 with migrations, scratch repo secrets, and a protected Environment with a reviewer (paid plan or public repository). No live evidence is claimed.
Rollout blocker found (not caused by this PR; must be fixed before
@v2moves): blogcicd.yml, blogpreview-gc.ymland heaven-wwwcicd.ymlpassCLOUDFLARE_PREVIEW_API_TOKEN/CLOUDFLARE_PREVIEW_ACCOUNT_ID, which neither the current@v2(def3679) nor8ad4d40declares (commit5def01donmain). GitHub rejects such a caller before any job starts — reproduced asstartup_failurein 41599151750 (8ad4d40) and 41600536212 (fd6ab85); the run page readsInvalid secret, CLOUDFLARE_PREVIEW_API_TOKEN is not defined in the referenced workflow.(same forCLOUDFLARE_PREVIEW_ACCOUNT_ID); heaven-wwwCI/CD(41017574023) and blogPreview GCnightly runs show matchingstartup_failures (cause inferred, not read from a log). Remedy: drop those two lines from the three callers, or declare them as optional secrets. Fix PRs (open, not merged): blog#461, heaven-www#56; replicas of the fixed callers start against both thev2tag and this PR (fail closed only on missing scratch credentials). Preconditions, procedure and rollback for the tag move:docs/cf-worker-v2-rollout.md.Verdict: no merge blocker found in the implementation. Remaining items are
@v2-rollout requirements: the secrets fix, administrator Required-Check confirmation/update, and live non-production evidence.Detailed evidence: architecture/refactor guide · unified delivery + P1 fix · OpenSpec.
Compatibility and rollout requirements
<caller-job> / Lintto<caller-job> / CI / Lint(likewise for other checks). Audit every consumer's branch protection/rulesets before moving@v2; missing admin/ruleset visibility is unverified, not proof that no rule exists.$/requires GitHub Actions runner >= 2.336.0. The tested orglinuxrunner is newer; check all consumers' runner choices.$/is not verified for GHES.Review / rollout checklist
8ad4d408ad4d40(including stale Migration fail-closed on a real runner)mainand record the immutable merged SHA only after approval@v2(BLOCKED: none available)CLOUDFLARE_PREVIEW_*secrets before moving@v2(PRs open; merging runs the real Production pipelines)@v2/ migrate consumers in a separately approved rolloutDo not merge, move
@v2, or change consumers as part of this PR-description update.