Skip to content

ci: mutate only the changed files in the gate's mutation step - #582

Merged
dmealing merged 1 commit into
mainfrom
fm/mo-gate-mutation-diff-only
Nov 14, 2026
Merged

dmealing merged 1 commit into
mainfrom
fm/mo-gate-mutation-diff-only

Conversation

@dmealing

Copy link
Copy Markdown
Member

What

The ts-fast lane's mutation step (completeness-gate) ran the full Stryker pass over all six stryker.conf.mjs mutate files on every run. That includes every no-mistakes gate Test step. This PR scopes that step to the files the change actually touches.

  • Diff-scoped step (gate_completeness_diff, in --only ts-fast, --only ts, --quick and so the gate's Test step):
    1. Reads the mutate list from stryker.conf.mjs, which stays the only copy.
    2. Diffs it against git merge-base HEAD <base>. The base is chosen in this order: MO_CI_MUTATION_BASE; origin/$GITHUB_BASE_REF on a PR; otherwise whichever of origin/main and origin/release/* HEAD is fewest commits ahead of. That way a release/1.2 branch diffs against release/1.2.
    3. Runs stryker run --mutate <changed files>. Thresholds and concurrency are unchanged.
    • No listed file changed: it prints ⊘ SKIP mutation: none of the 6 files … changed since <base> (merge base <sha>).
    • No base resolves, or stryker.conf.mjs itself changed: it says so and runs the full pass.
  • The full pass is kept and runs in:
    • the flagless run's mutation lane
    • a new --only mutation section, for release prep and on demand
    • conformance.yml (tag v*, nightly, dispatch) through MO_CI_MUTATION_SCOPE=full
  • Sandbox: when DEV_SCRATCH names a directory, Stryker's sandbox goes there instead of the package's .stryker-tmp/, and it is removed when the step ends.

Why --mutate on the diff rather than Stryker incremental mode: incremental mode keeps a results file between runs. A cache shared across branches and worktrees could report a mutant as killed by tests that only another branch has. The diff scope keeps no state.

Effect on local-ci.yml: a push to main now SKIPs the step, because HEAD is the base and there is nothing new to mutate. The hosted conformance.yml nightly still runs the full pass every night.

Timing

Before, from seven recorded metaobjects gate Test steps on 2026-11-13 and 2026-11-14 (no new runs were made to measure this):

run mutation step Test wall
01M7CWAB… 9m36s 15m27s
01M7BY45… 14m51s 26m20s
01M7BEGP… 15m10s 20m14s
01M7D315… 18m36s 25m45s
01M7B771… 28m10s 38m42s
01M7BMVJ… 37m28s 46m57s
01M7BMWP… 43m39s 59m30s

After, the step run locally on this branch. The host load average was around 50.

case step time
none of the six files changed 0s, SKIP printed
one file changed (provider.ts, 66 mutants) 68s, score 62.12 (break threshold 35)

Validation

  • MO_CI_LIST_ONLY=1 scripts/ci-local.sh --only ts-fast shows: ts build + typecheck, conformance: typescript, completeness-gate (mutation, changed files).
  • --only mutation shows the full step.
  • --only ts-fast --only mutation shows the full step in place of the diff step, not in addition to it.
  • The flagless run's lane mutation shows the full step.
  • The no-base fallback and the MO_CI_MUTATION_SCOPE=full path were checked with the Stryker call stubbed out. Both print why and take the full path.
  • scripts/test-ci-ports-to-run.sh and scripts/test-ci-affected-ports.sh pass.
  • The full six-file pass was not re-run: its code path (gate_completeness) is unchanged apart from the sandbox location.

Delivered directly as a PR, without the no-mistakes gate, because gate runs are paused until this lands.

The ts-fast lane (and so --quick, --only ts and the no-mistakes Test step)
ran the full Stryker pass over all six metadata files on every run: 10-45
minutes of CPU per gate round on a loaded host, most of that step's wall time.

The step now diffs the stryker.conf.mjs `mutate` list against the merge base
with the base branch (MO_CI_MUTATION_BASE, else origin/$GITHUB_BASE_REF, else
whichever of origin/main and origin/release/* HEAD is closest to) and runs
`stryker run --mutate` on only the changed files. When none changed it prints
a SKIP naming the base. When no base resolves, or stryker.conf.mjs itself
changed, it says so and runs the full pass. Thresholds and concurrency are
unchanged; the list stays in stryker.conf.mjs only.

The full pass stays: the flagless run's mutation lane, a new
`--only mutation` section for release prep and on demand, and conformance.yml
(tags, nightly, dispatch) via MO_CI_MUTATION_SCOPE=full. Stryker's sandbox goes
under DEV_SCRATCH when that names a directory, and is removed afterwards.
@dmealing
dmealing merged commit 0cd1133 into main Nov 14, 2026
9 checks passed
@dmealing
dmealing deleted the fm/mo-gate-mutation-diff-only branch November 14, 2026 22:53
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant