feat(ci): promote mutation testing and the web e2e suite to required checks

.github/workflows/ci.yml

@@ -135,7 +135,7 @@ jobs:
 
   test-e2e:
     name: E2E test (web)
-    # Not yet a required check, matching mutation.yml's own precedent: the suite is new (ExaDev/documents.js#932) and hasn't run unattended in CI before, so it needs a run of real-world flakiness data before gating merges on it.
+    # Promoted to a required check (ExaDev/documents.js#1194): the suite has run unattended on every pull request and main push since it landed (ExaDev/documents.js#932), and its only observed red run tracked a regression on that pull request's own branch and cleared when the branch was fixed -- a genuine catch, not flakiness, which is exactly the run history gating merges on it requires. Requiredness itself lives in the repository ruleset ("main required checks"), not in this file; this job always runs on both triggers, so requiring it can never strand a pull request waiting for a check that does not report.
     runs-on: ubuntu-latest
     timeout-minutes: 20
     steps:
@@ -704,9 +704,18 @@ jobs:
 
   deploy-site:
     name: Build and deploy the web UI to Pages
-    # After release, so the deploy is built from the release commit the orchestrator just pushed (it bumps packages/web/package.json and tags it). For a commit that releases nothing, no new tag is created and this builds the current tip of main.
+    # After release, so the deploy is built from the release commit the orchestrator just pushed (it bumps packages/web/package.json and tags it). For a commit that releases nothing, no new tag is created and this builds the current tip of main. test-e2e is in the gate so a main-branch e2e failure blocks deploying the site the suite drives a real browser through, matching the required role e2e already plays on pull requests via the ruleset.
     needs:
-      [commitlint, lint, typecheck, test, test-workers, test-smoke, release]
+      [
+        commitlint,
+        lint,
+        typecheck,
+        test,
+        test-workers,
+        test-smoke,
+        test-e2e,
+        release,
+      ]
     if: github.ref == 'refs/heads/main' && github.event_name == 'push'
     runs-on: ubuntu-latest
     timeout-minutes: 20

.github/workflows/mutation.yml

@@ -1,6 +1,6 @@
 name: Mutation testing
 
-# Repo-wide, sharded, incrementally-cached mutation testing via Stryker. Deliberately not yet a required check (see the ruleset docs / repo-setup skill's stryker-mutation-testing reference): no package here has a measured thresholds.break yet, and stryker.shared.ts's packageStrykerConfig sets none, so this job reports real mutation scores without ever failing the build on them -- the correct default until a real baseline exists to gate against. Promote it once scores are measured and a break threshold is deliberately chosen per package.
+# Repo-wide, sharded, incrementally-cached mutation testing via Stryker. Each package is gated by its own stryker.config.ts break threshold (derived from a completed run's measured baseline -- see stryker.shared.ts's breakThreshold), and the mutation-result job at the bottom aggregates the shards into the one stable check context a branch ruleset can require: a required status check must report on every pull request, and the shard matrix's width varies with the affected set, so no individual shard context can be required without stranding a pull request that produced fewer shards than the ruleset names. Requiredness itself lives in the repository ruleset ("main required checks"), never in this file -- a pull request can only prepare the job the ruleset points at, and the job is only ready to be required once every package completes a green run behind its threshold.
 on:
   pull_request:
   push:
@@ -88,3 +88,28 @@ jobs:
           name: mutation-report-shard-${{ matrix.index }}
           path: packages/*/reports/mutation/mutation.html
           if-no-files-found: ignore
+
+  # The single check context to list as required in the repository ruleset. The mutation-test job above is a matrix whose width the plan job computes per run from the affected package set (three affected packages produce three shards, not eight), so requiring any shard context by name would leave a pull request that produced fewer shards forever waiting on a check that never reports. This job always runs -- `if: always()` keeps it reporting even when planning fails or produces no packages, where the matrix job is skipped -- and derives one conclusion from the outcomes it aggregates, so it is stable across every shard count.
+  mutation-result:
+    name: Mutation testing result
+    needs: [plan, mutation-test]
+    if: always()
+    runs-on: ubuntu-latest
+    timeout-minutes: 5
+    steps:
+      - name: Aggregate the shard outcomes into one conclusion
+        # A plan failure is a real failure (nothing can be said about the mutation state); a plan that found no affected packages is a pass (nothing to mutate); anything else gates on the matrix's own aggregate result, which is failure when any shard failed (a package fell below its break threshold or its run errored).
+        run: |
+          if [ "${{ needs.plan.result }}" != "success" ]; then
+            echo "::error::Mutation shard planning failed; see the Plan mutation shards job."
+            exit 1
+          fi
+          if [ "${{ needs.plan.outputs.has-packages }}" != "true" ]; then
+            echo "No mutation-affected packages in this change; nothing to gate."
+            exit 0
+          fi
+          if [ "${{ needs.mutation-test.result }}" != "success" ]; then
+            echo "::error::One or more mutation shards failed: a package's score fell below its break threshold, or its run failed."
+            exit 1
+          fi
+          echo "Every mutation shard succeeded."

README.md

@@ -107,8 +107,8 @@ pnpm test:coverage    # the same suites with coverage
 pnpm test:workers     # the same code inside workerd, the real Cloudflare Workers runtime
 pnpm test:smoke       # each package's built dist/ exercised as a real artifact
 pnpm test:corpus      # the real-world conformance corpora (gitignored, so local only)
-pnpm test:mutation    # Stryker mutation testing, sharded and cached in CI; not yet a required check (no package has a measured break threshold yet)
-pnpm test:e2e         # the web UI driven through a real browser (Playwright); not yet a required check (new suite, no unattended CI run history yet)
+pnpm test:mutation    # Stryker mutation testing, sharded and cached in CI, each package gated by the break threshold derived from its measured baseline (per-package stryker.config.ts); a required check once every package completes a green run behind its threshold
+pnpm test:e2e         # the web UI driven through a real browser (Playwright); a required check (named in the repository ruleset alongside Lint/Test and siblings)
 ```
 
 Every one of these runs through turbo, so a package whose inputs have not changed replays a cached result rather than re-running.
@@ -163,7 +163,7 @@ Every alias name's trusted publisher is registered against this repository and w
 
 ## CI
 
-`.github/workflows/ci.yml` holds one job per task — Commitlint, Lint, Typecheck, Test, Test (workerd), Smoke test — each running that task once across the workspace through turbo, followed by Release, its three post-release republish/attestation matrices (see Releases above), and the web UI's Pages deploy on `main`. On a pull request every turbo task runs with `--affected`, restricting work to the packages the branch changed and their dependents; on `main` the full workspace runs, so the caches later runs restore from are complete and the release gate covers everything. Each job restores turbo's cache keyed by task, so an unchanged package costs a cache replay rather than a rebuild. The Typecheck job additionally runs `attw --pack` across every published package after building it, checking that each package's declared types resolve under every module resolution mode — the web UI is excluded, since it publishes nothing and exposes no types.
+`.github/workflows/ci.yml` holds one job per task — Commitlint, Lint, Typecheck, Test, Test (workerd), Smoke test, E2E test — each running that task once across the workspace through turbo, followed by Release, its three post-release republish/attestation matrices (see Releases above), and the web UI's Pages deploy on `main`. Mutation testing runs in its own `.github/workflows/mutation.yml` (sharded, incrementally cached, serialised through one concurrency queue so it can never starve these jobs of runners), reporting one aggregate "Mutation testing result" check the repository ruleset can require once every package's run is green behind its threshold. On a pull request every turbo task runs with `--affected`, restricting work to the packages the branch changed and their dependents; on `main` the full workspace runs, so the caches later runs restore from are complete and the release gate covers everything. Each job restores turbo's cache keyed by task, so an unchanged package costs a cache replay rather than a rebuild. The Typecheck job additionally runs `attw --pack` across every published package after building it, checking that each package's declared types resolve under every module resolution mode — the web UI is excluded, since it publishes nothing and exposes no types.
 
 Dependabot covers the root manifest and every package's, batching minor and patch updates into one pull request and leaving majors individual; `.github/workflows/dependabot-auto-merge.yml` auto-merges the former once CI is green. The cross-repository `sibling-released` dispatch the separate repositories used to propagate version bumps between themselves is gone: the orchestrator does that inside a single run now, in dependency order, without a pull request per bump.
 

packages/archive-codec/stryker.config.ts

@@ -2,4 +2,6 @@ import { packageStrykerConfig } from "../../stryker.shared.ts";
 
 export default packageStrykerConfig({
   vitestConfigFile: "vitest.mutation.config.ts",
+  // First CI-measured baseline: 76.26% of 1954 valid mutants, timeout share 2.5% -- break = floor(score) minus the timeout share rounded up to whole points (minimum one), per the derivation rule on PackageStrykerOptions.breakThreshold.
+  breakThreshold: 73,
 });

packages/byte-codec/stryker.config.ts

@@ -1,3 +1,6 @@
 import { packageStrykerConfig } from "../../stryker.shared.ts";
 
-export default packageStrykerConfig();
+export default packageStrykerConfig({
+  // First CI-measured baseline: 63.99% of 636 valid mutants, timeout share 2.0% -- break = floor(score) minus the timeout share rounded up to whole points (minimum one), per the derivation rule on PackageStrykerOptions.breakThreshold.
+  breakThreshold: 60,
+});

packages/doc-codec/stryker.config.ts

@@ -2,4 +2,6 @@ import { packageStrykerConfig } from "../../stryker.shared.ts";
 
 export default packageStrykerConfig({
   vitestConfigFile: "vitest.mutation.config.ts",
+  // First CI-measured baseline: 71.12% of 3758 valid mutants, timeout share 1.8% -- break = floor(score) minus the timeout share rounded up to whole points (minimum one), per the derivation rule on PackageStrykerOptions.breakThreshold.
+  breakThreshold: 69,
 });

packages/document-cli/stryker.config.ts

@@ -8,4 +8,6 @@ export default packageStrykerConfig({
     "!src/**/*.test.tsx",
   ],
   vitestConfigFile: "vitest.mutation.config.ts",
+  // First CI-measured baseline: 33.33% of 6241 valid mutants, timeout share 0.02% -- break = floor(score) minus the timeout share rounded up to whole points (minimum one), per the derivation rule on PackageStrykerOptions.breakThreshold.
+  breakThreshold: 32,
 });

packages/document-compute.js/stryker.config.ts

@@ -1,3 +1,6 @@
 import { packageStrykerConfig } from "../../stryker.shared.ts";
 
-export default packageStrykerConfig();
+export default packageStrykerConfig({
+  // First CI-measured baseline: 64.48% of 518 valid mutants, timeout share 0.4% -- break = floor(score) minus the timeout share rounded up to whole points (minimum one), per the derivation rule on PackageStrykerOptions.breakThreshold.
+  breakThreshold: 63,
+});

packages/document-mcp/README.md

@@ -228,7 +228,7 @@ Every tool that takes or produces document bytes goes through the same two hybri
 | `odb_tables`                 | Lists every table an embedded `.odb` database declares — column names, types, and row data — across every storage tier `documents.js` supports (HSQLDB TEXT/CACHED/BINARY, Firebird gbak backups).                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                               |
 | `odb_forms`                  | Lists every form an `.odb` database declares, with each form's own data source and field-bound controls.                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                         |
 | `odb_reports`                | Lists every report an `.odb` database declares, with each report's own data-source command, band/group structure, and `rpt:` formula expressions.                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                |
-| `odb_query`                  | Runs a bounded `SELECT` (with optional JOINs of any kind, table aliases, a derived table in `FROM`, and `IN`/`EXISTS` subqueries) over an embedded `.odb` database's extracted tables — given directly as SQL or by naming a saved query. No database engine involved; no column aliases; an unsupported construct is reported as a tool error naming it, never silently ignored.                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                        |
+| `odb_query`                  | Runs a bounded `SELECT` (with optional JOINs of any kind, table aliases, a derived table in `FROM`, and `IN`/`EXISTS` subqueries) over an embedded `.odb` database's extracted tables — given directly as SQL or by naming a saved query. No database engine involved; no column aliases; an unsupported construct is reported as a tool error naming it, never silently ignored.                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                |
 | `odb_to_csv`                 | Extracts exactly one named table from an embedded `.odb` database as CSV bytes. The table name is required whenever the database declares more than one table.                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                   |
 | `odb_to_xlsx`                | Extracts every table an embedded `.odb` database declares into one xlsx workbook, one sheet per table.                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                           |
 | `odb_render_report`          | Resolves one of an `.odb` database's own reports — its data-bound command run through the bounded SQL engine, its `rpt:` formulas evaluated, its bands laid out — and renders the result to docx, odt, or pdf.                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                   |