Repository navigation
ci: mutate only the changed files in the gate's mutation step - #582
Merged
Merged
Conversation
The ts-fast lane (and so --quick, --only ts and the no-mistakes Test step) ran the full Stryker pass over all six metadata files on every run: 10-45 minutes of CPU per gate round on a loaded host, most of that step's wall time. The step now diffs the stryker.conf.mjs `mutate` list against the merge base with the base branch (MO_CI_MUTATION_BASE, else origin/$GITHUB_BASE_REF, else whichever of origin/main and origin/release/* HEAD is closest to) and runs `stryker run --mutate` on only the changed files. When none changed it prints a SKIP naming the base. When no base resolves, or stryker.conf.mjs itself changed, it says so and runs the full pass. Thresholds and concurrency are unchanged; the list stays in stryker.conf.mjs only. The full pass stays: the flagless run's mutation lane, a new `--only mutation` section for release prep and on demand, and conformance.yml (tags, nightly, dispatch) via MO_CI_MUTATION_SCOPE=full. Stryker's sandbox goes under DEV_SCRATCH when that names a directory, and is removed afterwards.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What
The ts-fast lane's mutation step (
completeness-gate) ran the full Stryker pass over all sixstryker.conf.mjsmutatefiles on every run. That includes every no-mistakes gate Test step. This PR scopes that step to the files the change actually touches.gate_completeness_diff, in--only ts-fast,--only ts,--quickand so the gate's Test step):mutatelist fromstryker.conf.mjs, which stays the only copy.git merge-base HEAD <base>. The base is chosen in this order:MO_CI_MUTATION_BASE;origin/$GITHUB_BASE_REFon a PR; otherwise whichever oforigin/mainandorigin/release/*HEAD is fewest commits ahead of. That way a release/1.2 branch diffs against release/1.2.stryker run --mutate <changed files>. Thresholds and concurrency are unchanged.⊘ SKIP mutation: none of the 6 files … changed since <base> (merge base <sha>).stryker.conf.mjsitself changed: it says so and runs the full pass.mutationlane--only mutationsection, for release prep and on demandconformance.yml(tagv*, nightly, dispatch) throughMO_CI_MUTATION_SCOPE=fullDEV_SCRATCHnames a directory, Stryker's sandbox goes there instead of the package's.stryker-tmp/, and it is removed when the step ends.Why
--mutateon the diff rather than Stryker incremental mode: incremental mode keeps a results file between runs. A cache shared across branches and worktrees could report a mutant as killed by tests that only another branch has. The diff scope keeps no state.Effect on local-ci.yml: a push to main now SKIPs the step, because HEAD is the base and there is nothing new to mutate. The hosted conformance.yml nightly still runs the full pass every night.
Timing
Before, from seven recorded metaobjects gate Test steps on 2026-11-13 and 2026-11-14 (no new runs were made to measure this):
After, the step run locally on this branch. The host load average was around 50.
provider.ts, 66 mutants)Validation
MO_CI_LIST_ONLY=1 scripts/ci-local.sh --only ts-fastshows: ts build + typecheck, conformance: typescript, completeness-gate (mutation, changed files).--only mutationshows the full step.--only ts-fast --only mutationshows the full step in place of the diff step, not in addition to it.MO_CI_MUTATION_SCOPE=fullpath were checked with the Stryker call stubbed out. Both print why and take the full path.scripts/test-ci-ports-to-run.shandscripts/test-ci-affected-ports.shpass.gate_completeness) is unchanged apart from the sandbox location.Delivered directly as a PR, without the no-mistakes gate, because gate runs are paused until this lands.