[anth-volk]

Fixes #801

Same-Repository Draft PR Check

• This PR is a draft.
• This PR is from a branch in PolicyEngine/policyengine-sim-api, not a personal fork.

Summary

Phase 1 step 1.8, group 8a: ruff's argument, idiom and pytest-style rules. Bottom of the stack since #799 merged. Group 8b (#804) follows with the complexity refactors and the switch to select = ["ALL"].

Suppression counts

	Inline suppression comments	Rule codes in them
main	231	244
This PR (85f690c)	221	225

Against main, this PR adds 2 suppression comments and removes 12:

• Added (2):
  • libs/policyengine-stage12-persistence/src/policyengine_stage12_persistence/store.py:157, ARG001 ("the ConnectionOpener protocol passes it by keyword"). This one is new; see "Suppressions this PR still adds".
  • stage12_runtime/fanout.py:241, BLE001. This is not new: the existing suppression at fanout.py:236 moved into the new per-call helper _cancel_segment_call.
• Removed (12):
  • 9 # noqa: BLE001, S110 comments on try: … except Exception: pass blocks, which now use contextlib.suppress(Exception) (see "Best-effort calls").
  • 2 BLE001 comments, in the JWT decoder and the gateway's job-status handler. These handlers now re-raise with from <caught error>, which BLE001 accepts.
  • The moved fanout.py:236.

Suppressions this PR still adds

• store.py:157 ARG001: open_connection_once is the default ConnectionOpener.
  • The store always passes operation_name= by keyword, and this implementation has no use for it.
  • Renaming the parameter would break that keyword.
  • The only signature ruff accepts is **_context: str, which accepts any keyword and so loosens the documented protocol.
  • Files under libs/*/src don't qualify for a file-level exception.

Changes

• Unittest exemption (decided for this group): the shared rules tool's tests use unittest on purpose, because the tool needs only the standard library.
  • scripts/rules/tests/** is exempt from ruff's pytest-style rules (PT), registered as sim.quality.unittest-ruff-exemptions.
• Rules: ruff.toml gains ARG, ASYNC, B, C4, FBT, PD, PERF, PGH, PLW, PT, PTH, RUF and SIM, one commit per project.
• Executor target version: ruff doesn't read the projects' own requires-python. Every project and library file was therefore linted with ruff's default target version (below 3.11), although the executor requires Python 3.13.
  • per-file-target-version now lists projects/policyengine-simulation-executor/** as py313.
  • That makes the PERF203 suppression in database_retry.py unnecessary: ruff applies PERF203 only below Python 3.11.
  • It also reports 8 findings, all fixed: 5 I001 (tomllib now sorts with the standard library) and 3 UP017 (datetime.UTC).
  • Other projects keep the default target version.
• Unused arguments (ARG):
  • Side-effect fixtures move to usefixtures, and unused *args/**kwargs become *_args/**_kwargs.
  • A named parameter is underscored only where every caller passes it positionally. That includes the executor's FastAPI lifespan(_app), which Starlette and FastAPI call positionally.
  • Tests take a file-level exception, decided 2026-10-16.
    • Test fakes and stubs mirror real signatures because callers pass by keyword: _build_simulation, _load_dataset, and stand-ins for Modal's FunctionCall.get(timeout=...) and JWTDecoder.__call__.
    • So ARG001, ARG002 and ARG005 join the file-level exceptions for tests and deployed-service tests.
    • At this head that covers 107 findings in 20 test files: ARG001 42, ARG002 34, ARG005 31.
  • Outside test directories, five former ARG suppressions are now code changes:
    • NoOpObservability subclasses the Observability protocol it implements, and span() is marked @override.
    • The executor's in-memory object store records each object's content type.
    • The gateway's mock FunctionCall records the timeout of each get().
    • The gateway's import_gateway_models fixture, which only requested the isolation fixture for its side effect, is gone. Its test uses usefixtures and imports the models module directly.
    • The executor lifespan's argument is underscored.
• Signatures (FBT):
  • JWTDecoder.__init__'s auto_error is keyword-only; its only caller already passed it by keyword.
  • Test fakes now match their real signatures, for example modal.Dict.from_name(..., *, create_if_missing).
  • The precompute app's Modal local_entrypoint takes main(*, force=False).
    • Modal 1.4.2 builds a command-line option for every non-variadic parameter and calls the entrypoint with keyword arguments, so --force and --no-force are unchanged.
    • .github/scripts/modal-precompute.sh already passes --force as an option.
• FastAPI auth (B008):
  • JWTDecoder.__call__ and the gateway's require_auth use Annotated[..., Depends(...)], so their token parameter no longer has a default; every caller passes one.
  • The entry app's CallerAuthenticator.__call__ keeps a module-level Depends marker, because FastAPI can't resolve Annotated on a callable instance in a module with postponed annotations.
  • Every OpenAPI document is byte-identical.
• Re-raised errors (B904, 10): all use from <caught error>.
  • They're in the JWT decoder, the gateway's auth dependency and route handlers, and the legacy-dict resolver; none is in Stage 12 code.
  • Responses and messages are unchanged; tracebacks now say "The above exception was the direct cause…".
• Idioms: comprehensions (C4, PERF401), pathlib (PTH), check=False on subprocess.run, function references instead of lambdas where nothing patches the target (PLW0108), ClassVar annotations and raw regex patterns (RUF), and pytest idioms (PT001, PT006, PT011 with match=, PT017, PT018).
  • Three findings are now code changes rather than suppressions:
    • SIM115: the executor's credentials temp file is entered on a contextlib.ExitStack held by a with statement around the yield.
      • It still stays open across the yield. The finally block still restores GOOGLE_APPLICATION_CREDENTIALS before the file is closed and deleted.
    • SIM118: the UK local-authority check calls HDFStore.keys(include="pandas"), which is the default call spelled out.
      • in store would also match HDF nodes that aren't pandas tables, so the explicit keys() call stays.
    • PT011: the native SPM test passes check= to pytest.raises (pytest 8.4), so the typed error code is part of the raises check.
• Best-effort calls (SIM105): nine blocks that ran telemetry, or a Modal call cancel, and swallowed any error with try: … except Exception: pass now use with contextlib.suppress(Exception):. The old reason is a comment above each.
  • Same behaviour: they suppress the same exceptions around the same calls.
  • Suppression comments: this removes 9 # noqa: BLE001, S110 comments (18 rule codes) that exist on main. SIM105 was not selected on main, so no SIM105 suppressions existed to remove.
  • Still catch-all: each block still catches every Exception. Ruff's BLE001, S110 and SIM105 do not check contextlib.suppress, so these nine catch-all blocks are not counted as suppressions.
  • Where: the entry app (3) and its auth (1), the Modal app (2), segmented national cleanup (1), and the gateway's auth (1) and endpoints (1).
  • Coverage comment: the change had rewritten the gateway auth comment # pragma: no cover as # Pragma: no cover; the lower-case spelling is restored. The comment sits on its own line above the with statement, so coverage.py excludes no statement because of it.
• Modal images:
  • Gateway and executor image files change, mostly in arguments and idioms.
  • In src/modal/app.py, two best-effort blocks become contextlib.suppress and import contextlib is added; it gains no __future__ import.
• Registry: sim.quality.ruff-lint now lists the 13 families. The rules tool is synced from the Public API's group 8a (Enable ruff argument, idiom and pytest-style rules (ARG, B, C4, FBT, PERF, PLW, PT, PTH, RUF, SIM and others) policyengine-api#3924). Both test-exemption rules gain ARG001, ARG002 and ARG005.

Commits added on 2026-10-20 (85f690c)

1. Lint the executor as Python 3.13, its required version: per-file-target-version, the PERF203 suppression removed, and the 8 I001/UP017 fixes.
2. Use or underscore the arguments behind five ARG suppressions.
3. Make the precompute entrypoint's force option keyword-only: removes FBT001/FBT002.
4. Replace the SIM115, SIM118 and PT011 suppressions with code.
5. Restore the lower-case coverage pragma comment in gateway auth.

Testing

At 85f690c, each project in an environment synced with uv sync --frozen --extra test --python 3.13:

Project	Result
executor	1,292 passed, 22 skipped, 3 deselected
entry	534 passed
gateway	243 passed, 1 deselected
fastapi	14 passed

• Not run: the three native SPM tests in test_canonical_spm_native.py are skipped without SPM_NATIVE_SMOKE_SOURCE, so the check= change there wasn't exercised. The integration tests need live services.
• Lint: ruff 0.15.22 check --extend-select RUF100 . and format --check projects/*/src libs/*/src are clean.
• Guards: the rules tool's unittests (80) and scripts/run_quality_guards.py, with the base main, pass. Rendering the rules leaves no diff.
• Golden OpenAPI: the gateway golden OpenAPI test passes, and the golden file is unchanged.
• CI/CD: no workflow changed.
  • No environment variable is added, renamed or rewired. The changed deployment code reads these:
    • setup_gcp_credentials reads GOOGLE_APPLICATION_CREDENTIALS, GOOGLE_APPLICATION_CREDENTIALS_JSON, GCP_CREDENTIALS_JSON, GOOGLE_CREDENTIALS and SERVICE_ACCOUNT_JSON, as before. The executor functions attach the gcp-credentials Modal secret, and Stage 12 workers attach stage12-evaluation-gcp-credentials. Both secrets exist; which keys they hold wasn't checked.
    • The precompute entrypoint reads POLICYENGINE_ARTIFACT_BUCKET, from the repository variable that the deploy workflow's "Run artifact precompute" step sets.
  • The other changed deployment files read no environment variable in the changed code: the executor's main.py, the two Modal utility programs, database_retry.py and the gateway auth comment.
• CI: this is the lowest unmerged PR in the stack, so its full CI runs, including the Modal image smoke.

🤖 Generated with Claude Code