[anth-volk]

Fixes #3980

Summary

Declares every managed v2 database role's table privileges in the repository, grants them on every deploy, checks them before the catalog is seeded, and runs the cross-database integration tests as those roles instead of the database superuser. This is the design agreed in #3980 (comment), written after #3977, where production's runtime role had no SELECT on legacy_policy_mappings and every reform save returned 503.

• Declared privileges: policyengine_api/data/v2/privileges.py has one entry per v2 table (plus alembic_version) for each of policyengine_v2_runtime, policyengine_v2_catalog_writer and policyengine_stage12_<environment>. "No access" is an explicit entry, not an omission. The migration role owns every table and isn't listed.
• Unit check of the code against the declarations (api.v2.runtime-privileges-declared): policyengine_api/data/v2/privilege_requirements.py reads each connector module with ast. reads* needs SELECT; creates*/updates*/deletes* need INSERT/UPDATE/DELETE plus SELECT; with_for_update needs UPDATE on the tables it locks. A test fails when a connector needs a privilege its role isn't declared to hold. The proof fixture repeats the v2/user-policies returns 503 for logged-in users: reforms cannot be saved #3977 mistake.
• Grant and check on deploy (api.v2.deployed-privileges-match): scripts/reconcile_v2_privileges.py apply runs in seed-v2-database.yml straight after the schema upgrade, then verify, before the catalog is seeded. Both connect with the migration credential, which owns every table. apply grants each declared privilege a role lacks, never revokes, reports privileges the file doesn't declare and the projects' default privileges in the job summary, and grants nothing if a declared role or table is missing. verify checks every declared privilege with has_table_privilege and exits 1 on any that is missing. The report names roles, tables and privileges only, never a connection URL.
• Integration tests as the real roles (api.v2.runtime-privileges-tested-as-runtime-role): v2-integration-check.yml creates the managed roles in the disposable database (--create-missing-roles is refused for any persistent database), grants and verifies their privileges, and then:
  • the identity persistence tests run the v2 user services as policyengine_v2_runtime (setup and cleanup stay on the superuser);
  • a new test runs the comparison-run services as policyengine_stage12_test, and checks that role can't read users;
  • a new test covers the reconcile operations: a second apply grants nothing, an extra privilege is reported and kept, and a missing role blocks every grant.
• Rules, docs and changelog: three entries in rules/registry.toml with regenerated docs/generated/rules.md and AGENTS.md; a "Table privileges" section in docs/engineering/skills/alembic-migrations.md; a paragraph in v2-code-organization.md; and a correction to docs/migration/stage-8-supabase-target.md, which said ordinary read/write was granted to the runtime role. The changes under rules/ need approval from a core developer other than the author (api.process.rule-machinery-code-owned).

What the first deploys will grant and report

Planned offline from each project's grants, read with read-only catalog queries on 2026-10-20. Both projects have every declared role and table.

Environment	Grants	Extra privileges reported (not revoked)
Staging	None	policyengine_v2_runtime: DELETE on legacy_policy_mappings and on legacy_household_mappings
Production	policyengine_v2_runtime: INSERT on policies, households and parameter_values; INSERT and UPDATE on legacy_policy_mappings and legacy_household_mappings	None

The production grants are a deliberate choice. The code that needs them is the Stage 10 dual-write copy into legacy_policy_mappings/legacy_household_mappings (off in production: DB_WRITE_POLICY=cloud_sql) and the writes behind the native POST /v2/policies and POST /v2/households routes (production sends these to the Flask implementation: ROUTE_IMPL_POLICY=flask_fallback). We chose to count those code paths, so the declarations match what the code can do in any configuration, rather than exclude them. UPDATE on the mapping tables comes from read_legacy_policy_mapping/read_legacy_household_mapping locking rows with FOR UPDATE. The two mapping tables are planned for removal; when they go, these declarations and the related rules entries will be narrowed again.

Production's current grants also include the SELECT on legacy_policy_mappings, legacy_household_mappings, policies and households that was applied by hand to fix #3977. That grant is now declared in the repository.

Decisions in the design

• Grants are applied by a script on every deploy, not written into Alembic revisions: revisions stay generated-only, the Stage 12 role's name differs per environment, and a revision runs once per database, so it can't repair drift.
• Extra privileges are reported, never revoked. A later change may switch to revoking and drop the projects' default privileges.
• Stage 12 connectors (services/v2/comparison_runs) are checked against policyengine_stage12_<environment>; every other connector against policyengine_v2_runtime. The catalog writer's privileges are copied from production, because its code isn't organized as connector modules; the deploy check covers them.

Known limits

• The unit check reads model names in connector modules, so it misses raw text() SQL and SQL issued outside database_connectors/. The integration tests as the real roles cover the user services and comparison-run services. The policy, household and dual-write integration tests still run as the superuser.
• The runtime role keeps read/write access to the two Stage 12 tables because production grants it, although no Public API route uses them. That's a candidate to narrow once revoking is allowed.
• Default privileges remain in both projects, so a new table still gives the runtime role full read/write until they're dropped. The apply report lists them.

Environment variables

Every variable the new steps and scripts read, its provider, and how it reaches them:

Variable	Read by	Provider	Wiring	Status
V2_MIGRATION_DATABASE_URL (seed job)	new apply and verify steps	GitHub environment secret in staging and production (both confirmed with gh secret list --env)	each new step's env: maps secrets.V2_MIGRATION_DATABASE_URL, as the existing steps do; push.yml calls seed-v2-database.yml with secrets: inherit for staging and for production	confirmed
V2_SUPABASE_PROJECT_REF	settings loader, to check the URL names the configured project	GitHub environment variable in staging and production (confirmed with gh variable list --env)	seed job env: (already present)	confirmed
V2_SUPABASE_ENVIRONMENT	same	GitHub environment variable in staging and production (confirmed)	seed job env: (already present)	confirmed
DEPLOYMENT_ENVIRONMENT (seed job)	run(): names the Stage 12 role; must be staging or production for a persistent database	deployment_environment input: literal staging/production in push.yml	seed job env: (already present)	confirmed
DEPLOYMENT_ENVIRONMENT (integration job)	the reconcile steps and the integration fixtures	literal test	added to the catalog-runtime-integration job env:	confirmed
V2_ALEMBIC_DISPOSABLE_TEST	settings loader; --create-missing-roles requires it	literal "1" in the integration job; absent from the seed job	integration job env: (already present)	confirmed
V2_MIGRATION_DATABASE_URL (integration job)	reconcile steps and integration tests	literal disposable URL	integration job env: (already present)	confirmed
GITHUB_STEP_SUMMARY	apply/verify report	GitHub Actions runner	automatic	confirmed

In a persistent database, run() calls qualify_v2_connection, which requires the connection's current_user to be policyengine_v2_migrator with CREATE on schema public. The existing schema-upgrade step already passes the same check with the same secret.

Inline suppressions

None added. privilege_reconciliation.py prints its report and runs as a program (if __name__ == "__main__":), so it joins the existing T201 file-level exception for such modules (api.quality.command-line-modules-ruff-exemptions), like the migration qualification modules.

Follow-up

The simulation API runs as policyengine_stage12_<environment> and needs its own check before deploys: PolicyEngine/policyengine-sim-api#855.

Testing

• uv run pytest tests/unit/v2/test_privilege_manifest.py tests/unit/v2/test_privilege_requirements.py tests/unit/v2/test_privilege_reconciliation.py -q: 70 passed.
• make test: 2,866 passed.
• python scripts/run_quality_guards.py: migration-contracts, rules-registry and ci-scripts passed. python scripts/export_migration_contracts.py: no changes.
• python -m scripts.rules guard --base-ref origin/master: passed (three new rules noted). python -m unittest discover -s scripts/rules/tests -t .: 80 passed.
• ruff format --check . and ruff check .: clean. make typecheck-v2: no issues; the new modules also pass mypy, though they aren't in [tool.mypy].files.
• Against a disposable PostgreSQL 17 (DEPLOYMENT_ENVIRONMENT=test): alembic upgrade head; apply --create-missing-roles created the three roles and granted every declared privilege; a second apply reported "Nothing: every declared privilege was already granted."; verify exited 0. After revoking the runtime role's SELECT on legacy_policy_mappings by hand, verify exited 1 naming it, and apply granted it back.
• The PostgreSQL integration tests (catalog publication, metadata routes, policy and household persistence, user-policy mirroring, identity persistence as the runtime role, comparison runs as the Stage 12 role, reconcile operations, and the Alembic lifecycle): 60 passed.
• The local environment uses Python 3.14; CI uses 3.12.

🤖 Generated with Claude Code

[codecov]

Codecov Report

❌ Patch coverage is 88.12500% with 38 lines in your changes missing coverage. Please review.
✅ Project coverage is 84.62%. Comparing base (1163955) to head (accc0f2).
⚠️ Report is 6 commits behind head on master.

Files with missing lines	Patch %	Lines
...licyengine_api/data/v2/privilege_reconciliation.py	84.57%	15 Missing and 14 partials ⚠️
policyengine_api/data/v2/privilege_requirements.py	91.57%	2 Missing and 6 partials ⚠️
policyengine_api/data/v2/privileges.py	97.29%	0 Missing and 1 partial ⚠️

Additional details and impacted files

@@             Coverage Diff             @@
##           master    #3985       +/-   ##
===========================================
+ Coverage   46.17%   84.62%   +38.44%     
===========================================
  Files         239      258       +19     
  Lines       13987    15684     +1697     
  Branches     2272     2642      +370     
===========================================
+ Hits         6458    13272     +6814     
+ Misses       6958     1637     -5321     
- Partials      571      775      +204     

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.