[anth-volk]

Fixes #3993

Summary

The Cloud Run image for the Public API installs from uv.lock (uv sync --frozen --no-dev --no-editable), but the CI test jobs installed with pip install -e ".[dev]", which resolves the newest releases on PyPI. For example, PR #3933's run 39629243016 installed SQLAlchemy 2.1.6 and psycopg 3.3.6 (the lock pins 2.0.49 and 3.3.4), and six tests failed only in CI. This PR makes every CI test job install from uv.lock and run its tests through uv run --frozen on Python 3.12, the version the Cloud Run image runs.

A separate PR adds a daily workflow that relocks uv.lock and opens a PR, so dependency upgrades still get tested in CI.

Python version

.python-version pins 3.14, so uv used 3.14 even after actions/setup-python installed 3.12. The Type-check API v2 job's log shows uv downloading CPython 3.14.8. Every job changed here sets UV_PYTHON: "3.12", and the Makefile exports UV_PYTHON ?= 3.12.

Install sites

Place	Before	After
make install	pip install -e ".[dev]"	uv sync --frozen --extra dev
make test, test-env-vars, debug-test, debug, debug-asgi	bare python -m coverage, pytest, flask, uvicorn	uv run --frozen --extra dev ...
make changelog (towncrier step)	bare towncrier	uv run --frozen --extra dev towncrier
pr.yml Test	make install (pip), make test	set up uv, make install (locked), make test (locked)
pr.yml Test environment variables	make install (pip), bare pytest	set up uv, make install, uv run --frozen --extra dev pytest
push.yml versioning	pip install towncrier (unpinned)	removed; build_changelog.sh runs uv run --frozen --extra dev towncrier after uv lock
push.yml staging deploy (release tests, staging integration tests)	make install (pip), bare python -m pytest	set up uv, make install, uv run --frozen --extra dev python -m pytest
push.yml parallel live staging tests	pip install pytest pytest-xdist httpx (unpinned)	set up uv, make install; run_live_staging_tests.sh uses uv run --frozen --extra dev
push.yml production candidate smoke tests	pip install pytest httpx (runner's system Python, unpinned)	set up Python 3.12 and uv, make install, uv run --frozen --extra dev python -m pytest
alembic-v1-check.yml	make install (pip), bare python -m pytest / python -m alembic	set up uv, make install, uv run --frozen --extra dev ...
alembic-v2-check.yml, v2-integration-check.yml	uv sync --frozen, uv run without --frozen, Python 3.14	uv sync --frozen --extra dev, uv run --frozen --extra dev, Python 3.12

pytest-xdist is added to the dev extra so the two-worker live tests can run from the lock. uv lock was run with uv 0.12.1, the version in the Cloud Run image, to keep lock revision 3; it adds only pytest-xdist 3.8.0 and execnet 2.1.2.

Left outside the lock

• migrate-v1-cloud-sql.yml and process-v1-household-mirror-event.yml now run an explicit unlocked pip install -e ".[dev]" (the command make install used to run), so they behave exactly as before. Their scripts run the bare python from actions/setup-python, and that interpreter doesn't see the .venv that make install now creates. Moving them to the lock means running their scripts through uv run and verifying their database credentials and Cloud SQL Auth Proxy wiring against real databases, so it is left for a separate, verified change. They still run against whatever versions are newest on PyPI, the same mismatch that caused Refactor the application package within ruff's complexity limits #3933's failure.
• pip install "shellcheck-py==0.11.0.1" (quality-guards jobs in pr.yml and push.yml) stays pinned. shellcheck-py is not a project dependency, and the quality-guards job runs only standard-library guard code without installing the project.
• install_locked_ruff.sh (lint jobs) still installs ruff with pip, but at the version read from uv.lock. ruff has no Python dependencies.
• pip install requests uv in update-policyengine-bundle.yaml is unchanged. It is a bot workflow that updates pins and relocks, not a test job.
• seed-v2-database.yml (operational, against real databases) and the Type-check API v2 job already sync from the lock but still run uv run without --frozen and on Python 3.14. Neither runs tests, so neither is changed here.
• docker/Dockerfile (the generic image) installs policyengine[models]==6.7.5 with pip and is out of scope.

Coverage and Codecov

make test still runs coverage run -a --branch -m pytest ... followed by coverage xml -i, now through uv run, from the repository root, so coverage.xml is written where codecov/codecov-action@v5 finds it. The coverage version doesn't change: policyengine-canada requires coverage<7, so both the old pip install (run 39629243016 built coverage-6.5.0-cp312 from the sdist) and the lock use coverage 6.5.0.

Environment variables

Each changed job's environment variables, with their provider. uv run passes the parent environment through to the test process unchanged. I checked this locally: values set before uv run, before make and before env -u ... make all reached the Python process, and the variable removed with env -u stayed removed.

• UV_PYTHON (new): a literal in each changed job's env:, plus the Makefile default.
• OLD_SIMULATION_GATEWAY_URL: repository variable, set at workflow level in pr.yml and push.yml.
• pr.yml Test environment variables and Test:
  • POLICYENGINE_DB_PASSWORD, CODECOV_TOKEN, GCP_WORKLOAD_IDENTITY_PROVIDER, GCP_DEPLOY_SERVICE_ACCOUNT: repository secrets.
  • POLICYENGINE_GITHUB_MICRODATA_AUTH_TOKEN, PE_UK_PRIVATE_HF_READ_TOKEN (exposed as HUGGING_FACE_TOKEN), OPENAI_API_KEY: organization secrets visible to this repository.
  • All are passed in step env:, as before.
  • Google credentials: google-github-actions/auth (workload identity) writes the credentials file and exports GOOGLE_APPLICATION_CREDENTIALS and related variables through GITHUB_ENV, so later steps, including uv run, inherit them.
• push.yml versioning: APP_ID and APP_PRIVATE_KEY are organization secrets, unchanged. towncrier reads no environment variables.
• push.yml staging deploy:
  • Release-test step: POLICYENGINE_DB_PASSWORD (repository secret); POLICYENGINE_GITHUB_MICRODATA_AUTH_TOKEN, OPENAI_API_KEY, PE_UK_PRIVATE_HF_READ_TOKEN, and GATEWAY_AUTH_ISSUER, GATEWAY_AUTH_AUDIENCE, GATEWAY_AUTH_CLIENT_ID, GATEWAY_AUTH_CLIENT_SECRET_RESOURCE (organization secrets); RUNTIME_CACHE_* (literals). run_release_tests.sh still removes the deployment settings with env -u, and UV_PYTHON is not among them.
  • Staging integration tests: API_BASE_URL and STAGING_API_TEST_PROBE_ID come from step outputs; V2_MIGRATION_DATABASE_URL, STAGING_AUTH0_TEST_CLIENT_ID, _CLIENT_SECRET, _USERNAME and _PASSWORD are staging environment secrets; STAGING_AUTH0_TEST_DOMAIN, _AUDIENCE and _REALM are staging environment variables.
• push.yml parallel live tests: API_BASE_URL and STAGING_API_TEST_PROBE_ID come from the outputs of the job it needs and run identifiers.
• push.yml production candidate smoke tests: API_BASE_URL and STAGING_API_TEST_PROBE_ID come from step outputs. The job-level variables are unchanged and come from the production environment and the repository.
• Alembic v1/v2 checks and cross-database tests: database URLs and flags are job-owned literals for disposable service containers. CODECOV_TOKEN is passed explicitly by the calling workflows, unchanged.

Not verified from here: whether each secret's value is current, since only the names can be listed. These variables are also used by changed jobs but aren't listed as repository, environment or organization variables: vars.POLICYENGINE_DB_USER and vars.POLICYENGINE_DB_NAME, used in the deploy steps, which this PR doesn't change.

Testing

tests/unit/test_locked_test_environments.py finds every workflow job that runs pytest or coverage, follows the .github/**/*.sh scripts and Make targets it runs, and requires that the job:

• installs nothing with pip;
• runs every pytest and coverage command through uv run --frozen;
• sets UV_PYTHON: "3.12".

It also checks:

• that the expected eight test jobs are found;
• that make install is uv sync --frozen --extra dev and that the Make test targets run through the lock;
• that the two operational jobs keep their explicit install.

Red-green results by commit:

1. Lock test added: 29 failed and 3 passed against origin/master. The failures were the Makefile commands, the pip and make install installs, bare pytest/python -m pytest, the missing UV_PYTHON, and the operational jobs still using make install.
2. Makefile and operational workflows: the 7 Makefile and operational tests pass.
3. pr.yml: the 6 pr.yml job tests pass.
4. push.yml: all push.yml job tests pass. The 6 failures left were the Alembic v1/v2 and cross-database jobs. The existing live-test assertion now expects make install. While doing this I found and fixed one detection error: the lock test was reading comment lines as executed commands.
5. Alembic and cross-database workflows: 32/32 pass. The existing Alembic v1 assertion that the workflow had no uv run now asserts the locked commands.
6. towncrier: a new test failed on pip install towncrier in the versioning job, and passes after the change.
7. Docs: README and testing guidance only, no test change.
8. Self-review fixes: the lock test module now uses a named tuple. It still fails 29/32 against origin/master and passes 32/32 at this commit. The README now lists the Make targets that run through uv run, instead of claiming all of them do.
9. Changelog script test: it now fakes uv. It failed in the full run, fails against the previous script, and passes now.

Other checks, all run in the locked environment (uv run --frozen --extra dev --python 3.12):

• ruff format --check . and ruff check . with the ruff pinned in uv.lock (0.15.12): pass.
• python scripts/run_quality_guards.py: migration-contracts, rules-registry and ci-scripts pass.
• python -m scripts.rules guard --base-ref origin/master: passes.
• python -m scripts.rules render: no changes.
• uv lock --check: consistent.
• No # noqa added.
• Full unit suite (make test, Python 3.12.13, the locked environment):
  • At commit 8: 1 failed and 3071 passed in 15:43. The failure was test_build_changelog_uses_the_pyproject_version, which put a fake towncrier on PATH, but the script now calls uv run. Commit 9 fakes uv instead. That test fails against the old script, and the module passes (10/10) at this commit.
  • The full suite was not rerun after commit 9.
  • coverage xml -i on the run's .coverage data wrote coverage.xml (coverage 6.5.0) in the repository root.

CI was not awaited because of the ongoing GitHub Actions incident.

🤖 Generated with Claude Code

[codecov]

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 86.00%. Comparing base (392e653) to head (3b12e1c).

Additional details and impacted files

@@             Coverage Diff             @@
##           master    #3995       +/-   ##
===========================================
+ Coverage   49.24%   86.00%   +36.76%     
===========================================
  Files         243      259       +16     
  Lines       14398    15777     +1379     
  Branches     2343     2628      +285     
===========================================
+ Hits         7090    13569     +6479     
+ Misses       6682     1456     -5226     
- Partials      626      752      +126     

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.