Skip to content

Commit eef660c

Browse files
committed
ci: mutate only the changed files in the gate's mutation step
The ts-fast lane (and so --quick, --only ts and the no-mistakes Test step) ran the full Stryker pass over all six metadata files on every run: 10-45 minutes of CPU per gate round on a loaded host, most of that step's wall time. The step now diffs the stryker.conf.mjs `mutate` list against the merge base with the base branch (MO_CI_MUTATION_BASE, else origin/$GITHUB_BASE_REF, else whichever of origin/main and origin/release/* HEAD is closest to) and runs `stryker run --mutate` on only the changed files. When none changed it prints a SKIP naming the base. When no base resolves, or stryker.conf.mjs itself changed, it says so and runs the full pass. Thresholds and concurrency are unchanged; the list stays in stryker.conf.mjs only. The full pass stays: the flagless run's mutation lane, a new `--only mutation` section for release prep and on demand, and conformance.yml (tags, nightly, dispatch) via MO_CI_MUTATION_SCOPE=full. Stryker's sandbox goes under DEV_SCRATCH when that names a directory, and is removed afterwards.
1 parent 1997f15 commit eef660c

4 files changed

Lines changed: 126 additions & 15 deletions

File tree

‎.github/workflows/conformance.yml‎

Lines changed: 4 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -18,6 +18,7 @@ name: conformance
1818
# gates — fixture-lint, doc-template drift, embedded-library drift, leak scan and
1919
# the other offline repo gates (`--only gates`)
2020
# ts-fast — workspace build + typecheck, TS conformance, completeness (mutation) gate
21+
# (the FULL pass here: MO_CI_MUTATION_SCOPE=full on that job)
2122
# csharp / java-fast (java + kotlin) / python — each port's conformance corpora
2223
# java-reactor — full-reactor `mvn clean install`, tests on (`--only java-slow`)
2324
# The Java lanes point the script's Maven repository at ~/.m2/repository so setup-java's
@@ -129,6 +130,9 @@ jobs:
129130
- name: Conformance (${{ matrix.lane }})
130131
env:
131132
LANE: ${{ matrix.lane }}
133+
# Tag, nightly and dispatch runs mutate EVERY stryker.conf.mjs file, not only the
134+
# changed ones ts-fast mutates elsewhere (see gate_completeness_diff).
135+
MO_CI_MUTATION_SCOPE: full
132136
run: |
133137
METAOBJECTS_CI_M2_REPO="$HOME/.m2/repository" \
134138
scripts/ci-local.sh --only "$LANE" --no-integration --strict-toolchains

‎.github/workflows/local-ci.yml‎

Lines changed: 3 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -116,8 +116,9 @@ jobs:
116116
# gate runs only named conformance FILES, so the rest of those suites ran nowhere
117117
# and a regression could sit red on `main` under a green ts lane.
118118
# Deliberately separate from ts-fast rather than folded into it: ts-fast carries
119-
# the 30-min mutation gate, which flakes under runner contention, and a mutation
120-
# flake must not mask a unit-test failure. The suites total ~5s, so this lane is
119+
# the mutation gate (diff-scoped: on a push to main it mutates nothing new and
120+
# SKIPs; the full pass runs in conformance.yml), which flaked under runner
121+
# contention, and a mutation flake must not mask a unit-test failure. The suites total ~5s, so this lane is
121122
# nearly free and gives an independent, fast signal.
122123
ts-unit:
123124
needs: detect

‎.no-mistakes.yaml‎

Lines changed: 6 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -11,8 +11,12 @@
1111
# along the script's own `--only` section boundaries so neither step repeats the other.
1212
# `scripts/ci-local.sh --help` enumerates what each section contains, and
1313
# `MO_CI_LIST_ONLY=1 scripts/ci-local.sh --only <section>` prints its exact step list.
14-
# Measured green on a warm tree 2026-09-18: test 256s (it carries the mutation gate and is
15-
# almost all of the cost), lint 23s.
14+
# Measured green on a warm tree 2026-09-18: test 256s, lint 23s. The test step's mutation
15+
# gate is DIFF-SCOPED (gate_completeness_diff in the script): it mutates only the
16+
# stryker.conf.mjs `mutate` files the branch changed against its base, and SKIPs when it
17+
# changed none. Before 2026-11-14 it ran the full pass every round, 10-45 min under load
18+
# and most of this step's wall time. The full pass is `--only mutation`, the flagless
19+
# run's mutation lane, and conformance.yml (tags, nightly, dispatch).
1620
#
1721
# ORDER IS LOAD-BEARING, and it is why the work is split this way round rather than one
1822
# `--quick` in `test`. The pipeline runs test BEFORE lint. The `gates` section contains

‎scripts/ci-local.sh‎

Lines changed: 113 additions & 11 deletions
Original file line numberDiff line numberDiff line change
@@ -21,7 +21,8 @@
2121
# scripts/ci-local.sh --quick # FAST inner-loop tier: the cheap TS-centric
2222
# # gates only (leak-scan, pom parity, fixture-
2323
# # lint, ts build+typecheck, ts conformance,
24-
# # completeness, drift). Skips the other-
24+
# # completeness on the CHANGED files only, drift).
25+
# # Skips the other-
2526
# # language ports, the reactor, and docker.
2627
# scripts/ci-local.sh --only <section> [--only <section> ...]
2728
# # Run one or more named sections. Mutually
@@ -31,10 +32,23 @@
3132
# # leak-scan → the public-repo leak scan ALONE
3233
# # (hygiene.yml's PR gate; also in gates)
3334
# # ts → ts build+typecheck, ts conformance,
34-
# # completeness-gate, full unit suites
35+
# # completeness-gate (changed files),
36+
# # full unit suites
3537
# # (ts-unit), integration-tests ts
3638
# # ts-fast → build+typecheck, conformance,
37-
# # completeness (the pure-code signal)
39+
# # completeness (the pure-code signal).
40+
# # Its mutation step mutates only the
41+
# # stryker.conf.mjs `mutate` files this
42+
# # change touches vs the base branch, and
43+
# # prints a SKIP when it touches none
44+
# # (MO_CI_MUTATION_SCOPE=full forces all;
45+
# # MO_CI_MUTATION_BASE sets the base ref)
46+
# # mutation → the FULL mutation pass over every
47+
# # `mutate` file (~10-45 min). For release
48+
# # prep and on demand; the flagless run
49+
# # and conformance.yml also run it. It
50+
# # replaces ts-fast's diff-scoped step
51+
# # when both are selected
3852
# # ts-unit → full per-package unit suites for
3953
# # metadata/render/runtime-ts + the
4054
# # client/web packages — runs in
@@ -136,8 +150,8 @@ while [ $# -gt 0 ]; do
136150
--no-integration) NO_INTEG=1 ;;
137151
--integration-only) INTEG_ONLY=1 ;;
138152
--only) shift; case "${1:-}" in
139-
gates|leak-scan|ts|ts-fast|ts-unit|ts-slow|java|java-fast|java-slow|python|csharp|cube) ONLY="$ONLY ${1}" ;;
140-
*) echo "--only expects gates|leak-scan|ts|ts-fast|ts-unit|ts-slow|java|java-fast|java-slow|python|csharp|cube, got '${1:-}'" >&2; exit 2 ;;
153+
gates|leak-scan|ts|ts-fast|ts-unit|ts-slow|mutation|java|java-fast|java-slow|python|csharp|cube) ONLY="$ONLY ${1}" ;;
154+
*) echo "--only expects gates|leak-scan|ts|ts-fast|ts-unit|ts-slow|mutation|java|java-fast|java-slow|python|csharp|cube, got '${1:-}'" >&2; exit 2 ;;
141155
esac ;;
142156
-h|--help) awk 'NR==1{next} /^set -uo/{exit} {sub(/^# ?/,""); print}' "$0"; exit 0 ;;
143157
*) echo "unknown arg: $1 (see --help)" >&2; exit 2 ;;
@@ -621,8 +635,9 @@ gate_conf_ts() {
621635
# conformance gate runs only NAMED conformance files in metadata/render, so the rest
622636
# of those suites (and runtime-ts + the client/web packages) ran nowhere — which is
623637
# how a @formExclude regression sat red on `main`. Deliberately its OWN lane rather
624-
# than folded into ts-fast: ts-fast carries the 30-min mutation gate, which flakes
625-
# under runner contention, and a mutation flake must not mask a unit-test failure.
638+
# than folded into ts-fast: ts-fast carries the mutation gate (diff-scoped since
639+
# 2026-11; the full pass took up to ~45 min and flaked under runner contention), and a
640+
# mutation flake must not mask a unit-test failure.
626641
# Measured ~5s total, so the independent signal is nearly free.
627642
# NOTE: per-package `bun test` only — a bare root `bun test` walks java/python/
628643
# csharp/fixtures and takes many minutes.
@@ -769,7 +784,87 @@ gate_java_reactor() {
769784
# themselves (two concurrent `mvn install`s clobber the shared module jars). When the reactor
770785
# step is not selected — `--integration-only` — this installs it without its tests.
771786
gate_java_install() { ( cd server/java && mvn -q -DskipTests -Djacoco.skip=true install ); }
772-
gate_completeness() { ( cd server/typescript/packages/metadata && bun run conformance:mutation ); }
787+
# ── The mutation (completeness) gate: two scopes over ONE file list ───────────
788+
# Stryker mutates the files listed in `mutate` in server/typescript/packages/metadata/
789+
# stryker.conf.mjs — the ONLY copy of that list; both scopes read it from there.
790+
#
791+
# FULL (gate_completeness): every listed file. ~10–45 min of CPU on a contended host.
792+
# Runs in the flagless full run's `mutation` lane, in `--only mutation` (release prep
793+
# and on demand), and wherever MO_CI_MUTATION_SCOPE=full is set — conformance.yml does,
794+
# so its tag (`v*`, the release gate), nightly and dispatch runs mutate everything.
795+
# DIFF (gate_completeness_diff): only the listed files this change touches. This is
796+
# the step in ts-fast / --quick / --only ts, and so in the no-mistakes gate's Test
797+
# step (.no-mistakes.yaml). It used to run the full pass on every gate round, which
798+
# was most of that step's wall time and most of the machine's load.
799+
#
800+
# "Touches" = differs between the merge base with the base branch and the working tree.
801+
# The base is MO_CI_MUTATION_BASE when set, else origin/$GITHUB_BASE_REF on a GitHub PR,
802+
# else whichever of origin/main and origin/release/* HEAD is fewest commits ahead of (a
803+
# branch cut from release/1.2 diffs against release/1.2, not main). When no base resolves
804+
# (a shallow checkout, no remote refs) the step says so and runs the FULL pass — it never
805+
# guesses a smaller scope. A change to stryker.conf.mjs itself also runs the full pass.
806+
# When none of the listed files changed it prints a SKIP banner naming the base and the
807+
# reason, and passes: there is nothing new to mutate, and the full lane still covers the
808+
# whole list. Thresholds and concurrency come from stryker.conf.mjs unchanged.
809+
#
810+
# Why --mutate on the diff rather than Stryker's incremental mode: incremental keeps a
811+
# results file between runs, and a cache shared across branches and worktrees can report
812+
# a mutant killed by tests another branch has. The diff scope keeps no state at all.
813+
#
814+
# Sandboxes: when DEV_SCRATCH names a directory (a RAM scratch area), Stryker's sandbox
815+
# goes there instead of the package's .stryker-tmp/, and is removed when the step ends.
816+
MUTATION_PKG="server/typescript/packages/metadata"
817+
stryker_run() { # stryker_run [stryker args...] — in the metadata package, sandbox in DEV_SCRATCH when set
818+
local tmp="" rc
819+
if [ -n "${DEV_SCRATCH:-}" ] && [ -d "${DEV_SCRATCH}" ]; then
820+
tmp="${DEV_SCRATCH%/}/stryker/metaobjects-$(printf '%s' "$ROOT" | cksum | cut -d' ' -f1)-$$"
821+
mkdir -p "$tmp" || return 1
822+
fi
823+
( cd "$MUTATION_PKG" && bun run conformance:mutation ${tmp:+--tempDirName "$tmp"} "$@" ); rc=$?
824+
[ -n "$tmp" ] && rm -rf "$tmp"
825+
return "$rc"
826+
}
827+
gate_completeness() { bun_install || return 1; stryker_run; }
828+
mutation_base() { # prints the base ref the diff scope compares against; empty when none resolves
829+
local ref best="" best_n="" mb n
830+
if [ -n "${MO_CI_MUTATION_BASE:-}" ]; then echo "$MO_CI_MUTATION_BASE"; return 0; fi
831+
if [ -n "${GITHUB_BASE_REF:-}" ]; then echo "origin/$GITHUB_BASE_REF"; return 0; fi
832+
for ref in origin/main $(git for-each-ref --format='%(refname:short)' 'refs/remotes/origin/release/*'); do
833+
git rev-parse --verify -q "$ref^{commit}" >/dev/null 2>&1 || continue
834+
mb="$(git merge-base HEAD "$ref" 2>/dev/null)" || continue
835+
n="$(git rev-list --count "$mb..HEAD")"
836+
if [ -z "$best_n" ] || [ "$n" -lt "$best_n" ]; then best="$ref"; best_n="$n"; fi
837+
done
838+
echo "$best"
839+
}
840+
gate_completeness_diff() {
841+
if [ "${MO_CI_MUTATION_SCOPE:-diff}" = "full" ]; then
842+
echo " mutation scope: FULL (MO_CI_MUTATION_SCOPE=full)"; gate_completeness; return
843+
fi
844+
local base mb files changed
845+
base="$(mutation_base)"
846+
if [ -z "$base" ] || ! mb="$(git merge-base HEAD "$base" 2>/dev/null)"; then
847+
echo " mutation scope: FULL — no merge base with '${base:-origin/main or origin/release/*}' resolves here (shallow checkout or no remote refs), so the changed files cannot be computed"
848+
gate_completeness; return
849+
fi
850+
files="$(cd "$MUTATION_PKG" && bun -e 'const c = (await import("./stryker.conf.mjs")).default; console.log(c.mutate.join("\n"))')" || {
851+
echo " ✖ could not read the mutate list from $MUTATION_PKG/stryker.conf.mjs" >&2; return 1; }
852+
if ! git diff --quiet "$mb" -- "$MUTATION_PKG/stryker.conf.mjs"; then
853+
echo " mutation scope: FULL — stryker.conf.mjs changed since $base (merge base ${mb:0:12})"
854+
gate_completeness; return
855+
fi
856+
# shellcheck disable=SC2086 # the mutate entries are plain paths, one per line
857+
changed="$(cd "$MUTATION_PKG" && git diff --name-only --relative "$mb" -- $files)"
858+
if [ -z "$changed" ]; then
859+
echo ""
860+
echo " ⊘ SKIP mutation: none of the $(echo "$files" | wc -l | tr -d ' ') files in stryker.conf.mjs 'mutate' changed since $base (merge base ${mb:0:12})."
861+
echo " The full pass runs in the flagless run's mutation lane, in --only mutation, and in conformance.yml."
862+
return 0
863+
fi
864+
echo " mutation scope: changed files only, vs $base (merge base ${mb:0:12}):"
865+
echo "$changed" | sed 's/^/ /'
866+
stryker_run --mutate "$(echo "$changed" | paste -sd, -)"
867+
}
773868
gate_doc_template_drift() {
774869
bash scripts/sync-doc-templates.sh \
775870
&& git diff --exit-code -- \
@@ -1090,7 +1185,8 @@ lane_ts() {
10901185
if want ts; then step_if bun "conformance: typescript" gate_conf_ts; fi
10911186
if want ts; then step_if bun "ts unit suites" gate_ts_unit; fi
10921187
}
1093-
lane_mutation() { if want ts; then step_if bun "completeness-gate (mutation)" gate_completeness; fi; }
1188+
# The full run's mutation lane is the FULL pass over every file stryker.conf.mjs lists.
1189+
lane_mutation() { if want ts; then step_if bun "completeness-gate (mutation, full)" gate_completeness; fi; }
10941190
# ts-slow's half: the two real-PG suites plus the TS integration suite.
10951191
lane_ts_integ() {
10961192
integ_ready || return 0
@@ -1188,13 +1284,19 @@ if [ "$PARALLEL" -eq 0 ]; then
11881284
run_gates_bun
11891285
# The ts port is split into two lanes so CI can run them as separate jobs (see
11901286
# .github/workflows/local-ci.yml): a FAST lane (build+typecheck, conformance,
1191-
# completeness — the pure-code correctness signal, no docker) and a SLOW lane
1287+
# completeness on the changed files — the pure-code correctness signal, no docker) and a SLOW lane
11921288
# (docker/sidecar integration). Isolating integration means a docker/sidecar
11931289
# hiccup no longer fails the code signal, and vice versa. `--only ts` still runs both.
11941290
if want_any ts ts-fast; then step_if bun "ts build + typecheck" gate_ts_build_typecheck; fi
11951291
if want_any ts ts-fast; then step_if bun "conformance: typescript" gate_conf_ts; fi
11961292
if want_any ts ts-unit; then step_if bun "ts unit suites" gate_ts_unit; fi
1197-
if want_any ts ts-fast; then step_if bun "completeness-gate (mutation)" gate_completeness; fi
1293+
# The DIFF-scoped mutation step (see gate_completeness_diff); `--only mutation` asks for
1294+
# the full pass instead, and gets it in place of this step rather than as well.
1295+
if [ -n "$ONLY" ] && want mutation; then
1296+
step_if bun "completeness-gate (mutation, full)" gate_completeness
1297+
elif want_any ts ts-fast; then
1298+
step_if bun "completeness-gate (mutation, changed files)" gate_completeness_diff
1299+
fi
11981300
run_drift_gates
11991301

12001302
if [ "$QUICK" -eq 1 ]; then

0 commit comments

Comments
 (0)