[anth-volk]

Fixes #893

Same-Repository Draft PR Check

• This PR is a draft.
• This PR is from a branch in PolicyEngine/policyengine-sim-api, not a personal fork.

Merge order: the staging stable traffic has been moved

This PR had to wait until the staging service's stable traffic was moved off Cloud Run's sample container (revision policyengine-simulation-entry-staging-00001-cuh). The promotion's stable-URL check reads the revision header from /health, and the sample container returns an HTML page with no such header. A check that still reached it would fail, restore the sample container and fail the beta deployment, which stops the production deployment.

Done on 2026-10-30: stable traffic now goes to policyengine-simulation-entry-staging-00099-vin, built from main at 4b09f76d, whose beta deployment and authenticated tests passed. The stable-URL smoke check passed against it.

The first promotion deletes about 97 old staging revisions. After a successful promotion, the existing clean-up job (cleanup_entrypoint_revisions) removes tags from older revisions and deletes every staging revision except the promoted one and the one it replaced. The staging service has 99 revisions today, so about 97 are deleted, the sample container among them. This is expected and accepted.

Summary

• Staging promotes each tested build from main. The beta job in simulation-deploy.yml and the staging job of the Stage 12 comparison path (deploy-stage12-staging in stage12-deploy.yml) now pass promote_entrypoint: ${{ github.ref == 'refs/heads/main' }}. A deployment from main, including the release commit's, therefore runs the same promote_entrypoint job as production:

  • an exact revision change;
  • the stable-URL smoke check;
  • the release-sentinel check where the existing stack deploys;
  • the authentication tests against the stable URL;
  • restoring the previous revision on failure.

  It then runs the same revision clean-up. The staging stable URL then serves the revision that passed the staging tests.

• A manual run from another branch doesn't promote. Both workflows accept workflow_dispatch from any branch, and the beta environment has no deployment-branch policy. Such a run deploys and tests its candidate revision, but leaves the staging service's Cloud Run traffic where it was and deletes no revisions, so the last revision deployed from main stays available to simulation-rollback.yml.

  • Promotion covers only the Cloud Run entrypoint. As before this PR, a branch run still deploys the branch's Modal services to the staging Modal environment and publishes their routing there: the gateway, executor and Stage 12 workers from the main deployment workflow, the Stage 12 workers from the Stage 12 workflow. A later deployment from main replaces the apps it deploys, but routes and versions it doesn't deploy can stay.
  • The check reads the branch's own copy of the workflow, so it stops an accidental branch run from promoting, not a branch that edits it. Only a branch policy on beta would enforce it, and that would also stop branch runs from deploying to staging at all.

• New rule sim.deploy.staging-promotes-tested-revision (enforced, mechanical):

  • A test in test_deployment_assets_promotion.py reads every workflow, finds each job that calls the reusable deployment workflow with deployment_environment: staging, and fails unless it passes exactly promote_entrypoint: ${{ github.ref == 'refs/heads/main' }}. Both false and a literal true are violations.
  • It also requires the two known staging jobs to be found, so a renamed input cannot make the check pass by finding nothing.
  • Proof fixtures under rules/fixtures/sim.deploy.staging-promotes-tested-revision/: the bad fixture is the beta job as it was before this change. The proof test also rejects a literal true, and evaluates the required expression: true for refs/heads/main, false for another branch and for a tag.
  • docs/generated/rules.md and the Rules block in AGENTS.md are re-rendered.
  • This PR changes rules/, so it requests review from core-developers.

• Documentation:

  • README.md and the entrypoint's README.md and DEPLOYMENT.md now describe staging promotion, the revision clean-up and what a branch run does.
  • beta's value of SIMULATION_ENTRYPOINT_ENFORCE_PERMISSIONS now applies to every caller of the staging stable URL.
  • prod now also waits for beta's revision clean-up, so a failed staging clean-up stops the production deployment. A branch run skips both, and the prod environment, which accepts only main, refuses it.

• Changelog fragment changelog.d/893.changed.md.

• main is merged into this branch rather than rebased onto it, because other open branches are built on these commits.

Environment variables and secrets

The change only sets an input. The behavior that changes is that the reusable workflow's promote_entrypoint and cleanup_entrypoint_revisions jobs now also run for beta, on deployments from main. Their promotion result now also feeds finalize_routing and deployment_ready, as it already does for prod. Every value below reaches the job through the caller's secrets: inherit and the job's environment: ${{ inputs.release_environment }}, which is beta here. Values were checked by name through the GitHub API on 2026-10-30, after merging main; no secret value was read. Merging main changed neither job's variables.

Value	Read by	Provider	How it reaches the job	Status
SIMULATION_ENTRYPOINT_GCP_PROJECT_ID	promote_entrypoint (job env: PROJECT_ID), cleanup_entrypoint_revisions (job env:)	beta environment variable	vars. in job env:	Verified: set to policyengine-simulation-entry
SIMULATION_ENTRYPOINT_GCP_REGION	same two jobs	beta environment variable (falls back to us-central1)	vars. in job env:	Verified: set to us-central1
SIMULATION_ENTRYPOINT_GCP_WIF_PROVIDER, SIMULATION_ENTRYPOINT_GCP_DEPLOY_SERVICE_ACCOUNT	the google-github-actions/auth step in both jobs	beta environment secrets	secrets: inherit, then step with:	Verified by elimination: sim-entry-gh-deployer is the only service account in the project that this repository's GitHub workload identity pool may impersonate (roles/iam.workloadIdentityUser for attribute.repository/PolicyEngine/policyengine-sim-api), and it holds roles/run.admin at project level, which covers traffic changes and revision deletion. beta's deploy_entrypoint job already authenticates with these secrets.
SIMULATION_ENTRYPOINT_AUTH_ISSUER, SIMULATION_ENTRYPOINT_AUTH_AUDIENCE, SIMULATION_ENTRYPOINT_TEST_AUTH_CLIENT_ID, SIMULATION_ENTRYPOINT_TEST_AUTH_CLIENT_SECRET	the stable-URL sentinel and authentication steps of promote_entrypoint	beta environment secrets	secrets: inherit, then step env:	Verified present; they already serve beta's authenticated_test job against the candidate URL
SIMULATION_ENTRYPOINT_ENFORCE_PERMISSIONS	the stable-URL authentication step	beta environment variable	vars. in step env:	Not set in beta (same as today for the candidate), so the tests run in log-only mode
SIMULATION_ENTRYPOINT_CUSTOM_URL	promote_entrypoint (job env: CUSTOM_URL)	beta environment variable	vars. in job env:	Not set in beta, so the custom-hostname steps are skipped (production sets it)
MODAL_TOKEN_ID, MODAL_TOKEN_SECRET	finalize_routing (already ran for beta; it now also counts the promotion result)	repository and organization secrets	secrets: inherit, then step env:	Verified present; unchanged
Release sentinel artifact release-sentinel-beta	the stable-URL sentinel step	produced by beta's sentinel_expected job	actions/download-artifact	Already produced for beta; unchanged

No new variable or secret is introduced, and the deploy image, Cloud Run runtime configuration and Modal secrets are unchanged. Promotion only moves traffic to a revision that was already deployed with its own runtime environment.

Testing

After merging main (4b09f76d) and restricting promotion to main, all local:

• main split test_deployment_assets.py into smaller files, so the two new tests now sit in test_deployment_assets_promotion.py and the registry names them there.
• Failing first: with the workflows still passing true, test_staging_deployments_promote_their_tested_entrypoint_revision and test_deployment_uses_gcloud_workflow_without_terraform failed as expected. After the workflow change, they and test_rule_proof_staging_promotes_tested_revision pass, each run by node ID.
• ruff@0.15.22 check . and ruff@0.15.22 format --check (CI's commands): clean.
• python scripts/run_quality_guards.py --base-ref origin/main: every guard passed, including rules-registry, ci-scripts and file-length.
• python -m scripts.rules guard --base-ref origin/main: passed.
• .github/scripts/check-changelog-fragment.sh against main: passed.
• actionlint on both changed workflows: clean.

Earlier, on 8d259a5a: the same tests failed first and then passed; the entrypoint project's tests, the rules tool's tests, ruff and the quality guards passed; documentation-only commits were followed by the quality guards and test_deployment_assets.py.

Not run locally: the deployment itself. The promotion and clean-up behavior is first exercised by the post-merge beta deployment.

Review

• Review comment of 2026-10-27:
  • Any branch can replace the staging stable URL: fixed. Both staging jobs pass ${{ github.ref == 'refs/heads/main' }}, and the rule checks for that expression. The documentation says what a branch run still does.
  • The Stage 12 staging deployment promotes a revision whose existing-stack routing it didn't test: not changed in this PR. For now the Stage 12 comparison path is a test path, not one staging callers rely on.
• Merge resolution: one structured review round of the merged result against this PR's original change found no bugs.
• Main-only promotion: three structured review rounds. The first two found no bugs; each asked for more exact documentation of what a branch run still changes, which the last two commits make. The third found no bugs. Left for a follow-up: the routing registry's latest pointer and the Stage 12 manifest's default version move only to a newer version, so after a branch run with a newer policyengine.py version, unversioned staging requests keep reaching that branch's apps once main deploys again; that predates this PR.
• Before the merge: one structured review round found one documentation inconsistency, fixed in 4f34bf0f.

After merge (operator checks)

1. The first post-merge beta deployment: "Promote and verify Cloud Run entrypoint" and "Remove superseded Cloud Run revisions" succeed.
2. The staging stable URL's header: curl -sSi <staging stable URL>/health shows x-policyengine-simulation-revision: policyengine-simulation-entry-staging-<number>-<suffix>.

🤖 Generated with Claude Code

[vahid-ahmadi]

Review at 4f34bf0f5b.

Verdict: right direction, and the rollback path holds. But it lets any branch replace the staging stable URL, which I'd close before merging.

1. must .github/workflows/simulation-deploy.yml:46, .github/workflows/stage12-deploy.yml:58: the description records this as a consequence; I'd close it rather than document it. Both workflows accept workflow_dispatch from any branch, and the beta environment has no deployment-branch policy (checked through the API: beta has none, prod allows only main). With promote_entrypoint: true, an unmerged branch's build becomes what the Public API's staging service calls. The clean-up then deletes every other staging revision except the one it replaced, so after two branch dispatches in a row the last main revision is gone and simulation-rollback.yml cannot reach it. Give the Simulation Entrypoint its own simulation schema, migration chain and database connection #898 deliberately stops non-main runs from changing the database; promotion should match. Either give beta a main-only branch policy for the promotion and clean-up jobs, or pass promote_entrypoint: ${{ github.ref == 'refs/heads/main' }}. In the second case the new rule would check for that expression instead of a literal true.
2. should stage12-deploy.yml:58: the Stage 12 staging path runs with deploy_existing_stack: false and run_full_integration: false. So it now promotes a revision whose v1 existing-stack routing was not tested in that run (EXPECTED_IDENTITY is none), onto the URL the Public API's staging v1 traffic uses. Production's Stage 12 job already does this, behind the typed confirmation. For staging, worth stating in DEPLOYMENT.md that a Stage 12 dispatch replaces what beta last tested, or limiting promotion to the main workflow.

Checked and fine: promote_entrypoint needs deploy_entrypoint and authenticated_test to succeed, so a candidate that failed its tests cannot be promoted; a failed stable-URL check restores the previous revision; the clean-up keeps that previous revision for rollback; both workflows share the simulation-deploy-main concurrency group; prod now also waits for beta's clean-up.

What I ran: fresh clone, uv run --frozen --extra test pytest tests/test_deployment_assets.py in projects/policyengine-simulation-entry: 128 passed. CI on this head is green. I read the two workflow changes and the promotion, clean-up and test jobs they switch on in simulation-deploy.reusable.yml, but not the promotion and restore shell scripts. The deploy itself only runs after merge.

🤖 Generated with Claude Code

[anth-volk]

Thanks for the review. Point 1 is fixed in 62ca535: both staging jobs now pass promote_entrypoint: ${{ github.ref == 'refs/heads/main' }}, and the rule checks for that expression. 6369bc4 and 261957b document what a branch run still deploys. Point 2 isn't changed here: the Stage 12 comparison path is a test path for now, so its staging promotion stays as it is.