[anth-volk]

Fixes #3930

Summary

Phase 1 step 1.8, group 8b, part 2 of 3: the rest of policyengine_api/, refactored within ruff's complexity limits. Stacked on #3932.

• All 89 findings in scope are fixed by refactoring, with no suppressions. This PR adds no inline suppression of its own; see "Inline suppressions". Before each refactor, a commit adds characterization tests that pin current behaviour; 249 in all. A new tests/fixtures/recording_session.py records every statement, compiled for PostgreSQL, and every ORM write, in order.
• ASGI shell (asgi_factory, complexity 34 and 113 statements):
  • Lifespan, exception handlers and router wiring move to module level.
  • Per-request work moves into named steps: request context, telemetry, the route call and the response.
  • A new test pins the exact order of runtime and logger calls.
• Flask routes:
  • Household route errors keep the old except precedence. get_household_under_policy and _calculate each catch the error types they expect in one clause, which a helper maps to the same 404 and 400 responses. A following except Exception returns the SPM error response when the error or its cause is an SPM error, and otherwise calls module_logger.exception() itself and returns the 500 response.
  • Saved-policy writes share configuration and failure steps. They copy committed events to v2 through _copy_saved_policy_events_to_v2, which Make dual-write log failed v2 copies instead of returning HTTP 503 #3959 added.
  • Simulation identity checks move into their own step.
  • View signatures are unchanged.
• v2 services and routes:
  • Household and policy failures map through ordered error tables (run_v2_operation).
  • Household document validation and v1 household translation are split into collection, record and membership steps.
  • ComparisonReportRecord.validate_state keeps its Pydantic signature, with its checks split into helpers.
  • The parameter-value listing route takes one frozen ParameterValueListingQuery dependency; its OpenAPI is byte-identical.
• Data layer:
  • Catalog extraction splits into verification, selection and normalization steps, called in the same order. _normalize_region takes a private _SourceRegion protocol that declares the two region attributes it reads directly, code and region_type; it reads the rest through getattr with a default, as before.
  • Staging row generators move to module level.
  • Legacy user-policy persistence takes one revision value; a new test pins every statement and write in order.
  • Report and simulation runs take NewReportOutputRun, NewSimulationRun and ReportRunPackageVersions.
• Migration logging and flags:
  • Route groups come from a path table.
  • A served request is logged from one ServedRequest value.
  • Explicit database sources are one value.
• Mirroring and other services:
  • Mirror outcome loggers take one outcome value, and saved-policy mirror collaborators are grouped.
  • GatewayAuthTokenProvider takes GatewayAuthCredentials.
  • Two dead pieces are removed: policy_service._create_new_policy, which had no callers, and an always-true lock flag.
• Magic numbers (25): each becomes a named constant or an HTTPStatus member, with messages unchanged.
• Reconciled with Update PolicyEngine bundle to 6.3.4 #3918 (merged while this was in progress):
  • The split catalog extraction's region helper uses Update PolicyEngine bundle to 6.3.4 #3918's _serialize_region_filter_value, so flat membership lists such as the England filter are stored as JSON arrays.
  • The pinned "unsupported filter value" case now uses a mapping, which is still refused, instead of a list, which Update PolicyEngine bundle to 6.3.4 #3918 now accepts.
  • Update PolicyEngine bundle to 6.3.4 #3918's literal 255 limit is now MAX_REGION_FILTER_VALUE_LENGTH, with the message text unchanged.

Inline suppressions

This PR adds no inline suppression of its own. It removes 2 and moves 3 existing ones:

• Removed: the FBT001, FBT002 on the FastAPI current query parameter in fastapi_routes/v2/metadata/parameter_routes.py, which the ParameterValueListingQuery dependency replaces.
• Two merged into one:
  • The BLE001 in fastapi_routes/v2/households/routes.py and the one in fastapi_routes/v2/policies/routes.py become one in the shared fastapi_routes/v2/errors.py, with the same reason (return a secret-safe typed error).
• Moved within their files, unchanged: the ANN401 on asgi_factory's wsgi_app parameter and the BLE001 around telemetry in migration_logging.py.

2026-10-20 update

Three commits on top of the branch:

1. 4315457e7 Catch expected household calculation errors by type (BLE001): the split had folded the household routes' specific except clauses into one except Exception, with a BLE001 suppression in each of the two handlers. The handlers now catch the expected types first and log the unexpected failures directly, as described under "Flask routes". Responses, status codes and log messages are unchanged, and the complexity rules still pass.
2. 18c8b0c4c Type the catalog region parameter with a private protocol (ANN401): _normalize_region(source_region: Any) and its ANN401 suppression are replaced by the _SourceRegion protocol.
3. ccf4298fe Replace the policy mirror where it is defined in the user-policy persistence test: Enable ruff structure and import rules (INP, PLC, SLF, N, A, FAST, E) #3920 made mirror_legacy_user_policy_in_session import mirror_legacy_policy_in_session when it runs, because the policies services module is a banned module-level import. The new characterization test still replaced the function as an attribute of the user-policies services module, so all 13 of its tests errored at setup. Its fixture now replaces the function in the policies services module.

Lines containing # noqa in Python files (git grep -c '# noqa' -- '*.py', summed):

	Parent (#3932)	This PR
Before 2026-10-20	126	126 (3 added: 2 BLE001, 1 ANN401)
Now (after #3959)	98	96

2026-10-20: rebased onto #3959

#3959 makes dual-write log a failed v2 copy instead of returning 503. It moved the saved-policy copy into its own helper, _copy_saved_policy_events_to_v2, which logs instead of returning a response. Two of this PR's commits conflicted with it:

• 320ca6558 Share the saved-policy write configuration and failure steps:
  • This PR's 503-returning _mirror_user_policy_events is dropped, and the shared steps call Make dual-write log failed v2 copies instead of returning HTTP 503 #3959's helper.
  • The shared write configuration, the update-field check, the committed-update step, _UserPolicyWriteAttempt and the named 503 comparison are kept.
  • The ID bound is now MAXIMUM_LEGACY_INTEGER_ID, because Make dual-write log failed v2 copies instead of returning HTTP 503 #3959's _safe_legacy_integer_id also checks policy IDs.
  • The commit message says so.
• 74d912c20 Group saved-policy mirror collaborators: the grouped UserPolicyMirrorCollaborators is now passed in Make dual-write log failed v2 copies instead of returning HTTP 503 #3959's helper.
• The pin commit (f2ea2c97a): test_committed_write_without_a_mirror_is_mirror_unavailable asserted the 503 that Make dual-write log failed v2 copies instead of returning HTTP 503 #3959 removed. Its six cases are now test_committed_write_without_a_v2_copy_is_logged_and_still_succeeds, which captures Make dual-write log failed v2 copies instead of returning HTTP 503 #3959's behaviour:
  • the write returns its usual 201 or 200;
  • nothing secret reaches the response;
  • the route logs missing_mirror_revision or unexpected, and logs nothing extra when the mirroring service has already logged the failure.

Checks at this tip (ccf4298fe): pytest tests/unit gave 3,130 passed, 0 failed. Ruff, the quality guards and the rules guard are clean, and rendered rules are fresh.

Testing

2026-10-20 update. At this tip (4fab246dc), on Python 3.12 with the uv.lock dependencies (uv sync --frozen --extra dev --python 3.12):

• pytest tests/unit: 3,120 passed. Before the test fix in 4fab246dc, the same run gave 3,107 passed and the 13 setup errors described above; they also occur at the previous tip, 1ff10c1ba.
• The household route characterization tests (tests/unit/routes/test_household_route_error_responses.py, 15) and the catalog extraction tests pass.
• ruff format --check . and ruff check . (ruff 0.15.12) are clean, including C901, PLR0911, PLR0912 and PLR0915 on routes/household_routes.py.
• python scripts/run_quality_guards.py and python -m scripts.rules guard --base-ref origin/ruff-complexity-economy pass, and python -m scripts.rules render leaves no diff.
• mypy (the configured files): no issues. mypy on routes/household_routes.py and data/v2/catalog/extraction.py reports the same messages as before these commits.

Earlier notes:

• The make test set at this branch's tip: 2,848 passed, which is the base plus part 1's 166 and this PR's 249.
• Behaviour comparisons against the base:
  • Parameter-value queries are identical.
  • 4,800 household translation and normalization outcomes are identical.
  • 63 catalog extraction mutations are identical.
• make typecheck-v2 is clean. mypy on the 28 changed application files outside its file list gives identical output to the base.
• ruff check, ruff format --check and the quality guards (migration-contracts, rules-registry, ci-scripts) pass.
• The FastAPI OpenAPI, the /v2/openapi.json preview, the legacy spec and every existing model and adapter schema are byte-identical, with and without OLD_SIMULATION_GATEWAY_URL set.
• Not run locally: tests/integration/test_v1_user_policy_dual_write.py needs MySQL and PostgreSQL; its 6 call sites were updated.
• CI: above the bottom of its stack, so only CI complete runs until the PRs below merge.

🤖 Generated with Claude Code