ci(backend): parallelize full-suite coverage

.agent-loop/initiatives/WS-CI-001-backend-ci-acceleration/CHUNK_MAP.md

@@ -0,0 +1,10 @@
+# CHUNK MAP: WS-CI-001
+
+| Chunk | Title | Risk | Status |
+|---|---|---:|---|
+| `WS-CI-001-01` | Parallel Full-Suite Coverage | L1 | Implemented and internally reviewed; awaiting hosted PR proof/human review |
+| `WS-CI-001-02` | Safe Routing, Cache, and Timing Refinement | L1 | Future; not started and requires separate approval |
+
+Each chunk maps to one PR. Chunk 01 preserves the full suite and every coverage
+gate. Chunk 02 may be planned only from measured 01 evidence and cannot be named
+as 01's successor unless its reviewed contract is complete.

.agent-loop/initiatives/WS-CI-001-backend-ci-acceleration/DECISIONS.md

@@ -0,0 +1,39 @@
+# DECISIONS: WS-CI-001 - Backend CI Acceleration
+
+## 2026-07-20 - Prioritize CI acceleration before explicit-start automation
+
+The user selected backend CI efficiency as the next initiative after successful
+merge and live verification of `WS-ENG-001-04A`. `WS-ENG-001-04B` remains
+stopped and is not implicitly reprioritized by merge memory.
+
+## 2026-07-20 - Preserve the full suite
+
+The first implementation will reduce wall-clock time through isolated parallel
+execution, not by skipping tests or lowering coverage.
+
+## 2026-07-20 - Use file-level shards without a new scheduling dependency
+
+Test modules remain intact and are assigned deterministically from collected
+inventory. Shared-database xdist and third-party sharding services are rejected.
+
+## 2026-07-20 - Keep routing separate
+
+Path-based workflow routing, caches, and persistent runtime weighting require
+separate evidence and approval after parallel execution is proven.
+
+## 2026-07-20 - Preimplementation review repairs accepted
+
+Nine reviewer tracks required canonical filesystem and collected-node inventory,
+exact observed-node fan-in, coverage-byte hashing, checked-out-tree provenance,
+fixed artifact sets, explicit read-only permissions, digest-pinned PostgreSQL,
+stable `Backend / test`, canonical operator documentation, and a real local dry
+run. The plan and 01 contract now include those boundaries; all tracks pass.
+
+## 2026-07-20 - Bind nondeterministic parameter IDs safely
+
+The first hosted run proved that parameter display IDs containing import-time
+UUID values are not stable across preflight and shard processes. Raw preflight
+node IDs are therefore not executable cross-process authority. Shards execute
+validated whole modules, record final collection and completion in the same
+pytest process, require those exact sets to match, and bind their stable
+test-base cardinalities to the authenticated preflight manifest.

.agent-loop/initiatives/WS-CI-001-backend-ci-acceleration/DISCOVERY.md

@@ -0,0 +1,115 @@
+# DISCOVERY: WS-CI-001 - Backend CI Acceleration
+
+## Observed current behavior
+
+- `.github/workflows/backend.yml` defines one `test` job with a 240-minute
+  timeout, PostgreSQL 16, and a separately started pinned MinIO image.
+- `Backend full-suite coverage` invokes
+  `backend/scripts/run_isolated_tests.py` with a 12,600-second child timeout.
+- The runner creates a unique PostgreSQL role/database, migrates to the single
+  Alembic head, buffers child stdout/stderr, redacts database URLs, and cleans up.
+- Coverage is produced once and then read by twelve sequential `coverage report`
+  commands enforcing the repository-wide 78 percent floor and protected 90
+  percent subsystem floors.
+- `API contract real API e2e` provisions another isolated database after all
+  coverage reports.
+- The workflow runs for every pull request and every push to `main`.
+
+## Measured evidence
+
+PR #161 Backend run `29752233451`, job `88385435082`:
+
+| Step | Duration |
+|---|---:|
+| Runner/service initialization | 17 seconds |
+| Dependency installation | 20 seconds |
+| Lint and docstring coverage | 1 second |
+| Isolated runner self-test | 17 seconds |
+| Backend full-suite coverage | 25 minutes 31 seconds |
+| All twelve coverage reports | about 2 seconds |
+| API contract E2E | 9 seconds |
+| Total job | about 26 minutes 42 seconds |
+
+Recent successful Backend workflows have ranged from roughly 24 to 52 minutes.
+The full-suite process is therefore the dominant wall-clock cost.
+
+## Repository inventory
+
+- `backend/pyproject.toml` declares pytest, pytest-asyncio, pytest-cov, coverage,
+  and Ruff through the existing dev extra; it has no xdist or sharding package.
+- `backend/tests/` contains 31 `test_*.py` modules and approximately 999 direct
+  test functions before parameter expansion. Pytest's collected node-ID set is
+  the executable inventory and may be larger after parameter expansion.
+- The largest modules by source size are `test_auth.py`, `test_alembic.py`,
+  `test_projects.py`, `test_tasks.py`, and `test_authorization.py`.
+- Database-heavy modules construct async engines against the isolated database
+  URL. Migration and transaction tests make process-level database separation
+  safer than concurrent test processes sharing one database.
+- Artifact integration tests use the real MinIO endpoint exposed by the
+  workflow.
+
+## Existing tests and gates
+
+- `backend/tests/test_isolated_database_runner.py` proves owned database and
+  cleanup behavior.
+- `backend/tests/test_coverage_contract.py` protects coverage expectations.
+- `scripts/test_agent_gates.py` contains workflow-structure and CI-integrity
+  assertions and is the natural place for fail-closed workflow tests.
+- `.github/workflows/agent-gates.yml` separately validates merge intent, stale
+  wording, Markdown links, loop state, scope, and internal review evidence.
+- Branch protection currently receives the Backend job identity `test`; the
+  final fan-in must preserve that identity.
+
+## Dependencies and integrations
+
+- GitHub Actions hosted Ubuntu runners.
+- PostgreSQL 16 service container.
+- Pinned MinIO container image.
+- GitHub artifact upload/download actions, which must be pinned by immutable
+  commit SHA if introduced.
+- Existing coverage data format from pytest-cov/coverage.py.
+- MinIO is digest-pinned, but PostgreSQL currently uses the mutable
+  `public.ecr.aws/docker/library/postgres:16` tag. Immutable container provenance
+  is therefore a target for 01, not an existing property.
+
+## Gaps
+
+- No deterministic shard planner or checked shard manifest exists.
+- No exact-once filesystem-module or collected-node-ID proof exists across jobs.
+- No coverage-artifact provenance/fan-in validation exists.
+- No live per-shard timing evidence exists.
+- No current workflow test proves a failed/cancelled shard makes the final
+  required job fail.
+
+## Risks discovered
+
+- File size is only a weak runtime proxy; collected test count is a better
+  initial deterministic weight but still cannot perfectly balance slow modules.
+- `test_alembic.py` may remain a long-tail shard because modules must initially
+  remain intact.
+- Starting PostgreSQL and installing dependencies per shard increases aggregate
+  runner consumption while reducing elapsed time.
+- Coverage aggregation is trustworthy only if each artifact is bound to the
+  actual checked-out tree, expected shard ID, complete module/node inventory,
+  and SHA-256 digest of the exact coverage bytes.
+- A final job using unconditional execution must explicitly reject any upstream
+  non-success status; GitHub's default dependency skipping is not sufficient for
+  a stable required check.
+
+## Unknowns to measure during implementation
+
+- Exact collected node count per module after parameter expansion.
+- Actual shard balance on GitHub-hosted runners.
+- Whether all shards require MinIO or only the artifact shard.
+- Practical concurrency available to the repository/account.
+- Aggregate Actions-minute increase for four shards.
+
+## Conventions to preserve
+
+- Immutable action pins and the existing digest-pinned MinIO image; 01 must
+  replace the mutable PostgreSQL tag with a reviewed digest pin.
+- `run_isolated_tests.py` as the sole database ownership boundary.
+- Existing 78/90 coverage floors.
+- Backend required check named `test`.
+- Full exact-head hosted proof before merge.
+- No implementation until a reviewed chunk contract receives human approval.

.agent-loop/initiatives/WS-CI-001-backend-ci-acceleration/INTENT.md

@@ -0,0 +1,110 @@
+# INTENT: WS-CI-001 - Backend CI Acceleration
+
+## Problem being solved
+
+The required Backend GitHub Actions job runs the complete backend test suite in
+one sequential process. PR #161 spent 25 minutes 31 seconds in that single step
+and about 27 minutes overall even though it changed no backend product code.
+
+## Why this work matters
+
+Every implementation PR must retain full backend and subsystem coverage proof,
+but a 25-to-50-minute feedback loop slows review, repair, and safe delivery. The
+repository needs shorter wall-clock feedback without weakening its zero-trust
+CI evidence.
+
+## Current behavior
+
+- One Ubuntu job provisions PostgreSQL and MinIO.
+- One isolated database is migrated and used for all 999 collected tests across
+  31 test modules.
+- The full suite writes one coverage data file.
+- Twelve coverage reports and the API contract E2E run only after the full suite.
+- The isolated runner buffers child output, so the longest step exposes no live
+  progress.
+- PR #161 spent only 20 seconds installing dependencies; dependency setup is not
+  the primary bottleneck.
+
+## Target behavior
+
+- Every collected backend test still runs on every required Backend workflow.
+- Canonical filesystem test modules are assigned exactly once. Stable test-base
+  cardinality signatures bind runtime inventory to preflight, while each shard
+  proves its exact same-process collection equals its completions.
+- Each shard receives an independently owned migrated PostgreSQL database.
+- Coverage artifacts are combined centrally before the unchanged 78 percent
+  repository floor and 90 percent protected-subsystem floors are enforced.
+- The API contract E2E proof runs concurrently with test shards.
+- The final required Backend check fails closed on missing, duplicate, failed,
+  cancelled, malformed, foreign-tree, or byte-tampered shard evidence.
+- CI exposes shard progress and timing evidence.
+
+## Design chosen
+
+Use deterministic file-level sharding based on the collected test inventory.
+Keep each test module intact, distribute modules using collected test counts,
+run shards in independent GitHub jobs, upload uniquely named coverage artifacts,
+and combine them in one final required `test` job. File-level isolation avoids
+introducing unsafe concurrency inside a shared database and requires no new
+pytest scheduling dependency.
+
+## Alternatives considered
+
+- Skip the Backend workflow for engineering-loop PRs: rejected for the first
+  chunk because the user requires full-suite proof and path routing can hide an
+  incorrectly classified backend-affecting change.
+- Add `pytest-xdist` inside the existing job: rejected because tests share one
+  migrated database and many modules exercise transaction and migration state.
+- Hash node IDs across shards: rejected because it splits module fixtures and
+  makes database-sensitive module behavior harder to reason about.
+- Preserve one sequential process but add caching: rejected as the primary fix;
+  installation was 20 seconds while the suite was more than 25 minutes.
+- Lower or sample coverage: rejected as CI weakening.
+
+## Boundaries preserved
+
+- All tests, migrations, real PostgreSQL behavior, required MinIO behavior, API
+  contract proof, and coverage thresholds remain mandatory.
+- No backend application, schema, migration, authentication, authorization,
+  artifact, review, contribution, compensation, or payment behavior changes.
+- The final branch-protection check identity remains stable.
+- No third-party test-sharding service or production secret is introduced.
+- Human PR approval and existing Agent Gates remain unchanged.
+
+## Expected risks
+
+- A planner may omit or duplicate a test module.
+- Coverage artifacts may be mixed across commits or shards.
+- One oversized module may dominate a shard and limit speedup.
+- GitHub matrix cancellation may accidentally allow the aggregator to pass.
+- MinIO-dependent tests may be assigned to jobs without MinIO.
+- Coverage path differences may prevent correct combination.
+- GitHub concurrency and aggregate runner minutes may increase.
+
+## What must not change
+
+- No test may be skipped because of its shard assignment.
+- No assertion, coverage floor, required check, or failure propagation may be
+  weakened.
+- Shards must never share a database or database role.
+- Coverage from a different commit or incomplete shard set must not validate.
+- `WS-ENG-001-04B` remains inactive.
+
+## How this will be proven
+
+Unit tests will prove filesystem and stable node-signature inventory completeness,
+determinism, exact same-process collection/completion, stable ordering, malformed input
+rejection, and fail-closed coverage-manifest validation. Workflow review will
+prove isolated services and databases, immutable action pins, checked-out-tree
+binding, byte-hashed artifacts, complete fan-in, unchanged thresholds, stable
+required-check identity, and no skipped failure path. The implementation PR must
+run the old full suite locally where practical and the new hosted matrix on the
+exact checked-out tree; measured wall time will be compared with PR #161's
+25-minute-31-second full-suite baseline.
+
+## Human decisions required
+
+- Approve implementation of `WS-CI-001-01` after plan review.
+- Decide after hosted evidence whether four shards are the correct cost/latency
+  balance or should be adjusted in a later chunk.
+- Separately approve any later path-routing policy; it is not part of 01.