Skip to content

Commit 624a54c

Browse files
authored
ci: run only the lanes a change affects in local CI and the validation gate (#589)
* ci: run only the lanes a change affects in the gate's Test step Add scripts/ci-lane-map.txt, one path-to-lane table, and scripts/ci-select-lanes.sh, which reads it. ci-local.sh --affected lists the change since the merge base, prints the lanes it selected and skipped and the rule each path matched, and runs the selected ones; several ports run as parallel lanes like the full run, one stays serial. Unknown paths, an unreadable map and an unresolvable base all select every lane. ci-affected-ports.sh reads the same map, so the gate and the hosted workflows agree. docs/recipes is executed by port tests and no longer counts as docs. .no-mistakes.yaml: the test command is --affected, and test.instructions and test.path_instructions scope the evidence agent by the paths a change touches. * ci: treat docs pages as test inputs, share the base-ref rule, read the lane map once Review follow-ups for affected-lane selection: docs/ maps to every lane because port tests read those pages (only the root prose files select none), the selector CLI uses the same base rule as the gate, the map is parsed once per selection without per-line forks, and the selected-port lanes call the shared step functions directly. * no-mistakes(review): fix(ci): add --no-renames to port mapper; correct pr-ci docs comment * no-mistakes(test): apply changes made during test validation * no-mistakes(document): fix stale CI lane-selection doc claims
1 parent 8e52192 commit 624a54c

10 files changed

Lines changed: 629 additions & 94 deletions

‎.github/workflows/pr-ci.yml‎

Lines changed: 3 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -12,8 +12,9 @@ name: pr-ci
1212
# WHICH PORTS: `detect` maps the PR's changed paths to ports with
1313
# scripts/ci-affected-ports.sh, the same mapper local-ci.yml uses. A change under a
1414
# port's directory selects that port; a change to fixtures/, spec/, scripts/, .github/,
15-
# agent-context/ or anything unrecognised selects ALL ports (the mapper fails open); a
16-
# docs-only change selects none and runs only the offline `gates` lane.
15+
# agent-context/, docs/ or anything unrecognised selects ALL ports (the mapper fails
16+
# open); only the root prose files (README.md, CHANGELOG.md, CONTRIBUTING.md,
17+
# AGENTS.md, CLAUDE.md) select none and run only the offline `gates` lane.
1718
#
1819
# WHAT RUNS PER PORT: unit tests plus the conformance corpora, not the docker
1920
# integration suites (integration-tests.yml owns those, on tags and on demand):

‎.no-mistakes.yaml‎

Lines changed: 62 additions & 22 deletions
Original file line numberDiff line numberDiff line change
@@ -7,16 +7,23 @@
77
# conformance.yml's lanes, integration-tests.yml's Testcontainers matrix), so pinning the
88
# gate to it runs exactly the repository's checks, the same ones GitHub runs.
99
#
10-
# THE SPLIT. These two commands together are exactly `scripts/ci-local.sh --quick`, cut
11-
# along the script's own `--only` section boundaries so neither step repeats the other.
12-
# `scripts/ci-local.sh --help` enumerates what each section contains, and
13-
# `MO_CI_LIST_ONLY=1 scripts/ci-local.sh --only <section>` prints its exact step list.
14-
# Measured green on a warm tree 2026-09-18: test 256s, lint 23s. The test step's mutation
15-
# gate is DIFF-SCOPED (gate_completeness_diff in the script): it mutates only the
16-
# stryker.conf.mjs `mutate` files the branch changed against its base, and SKIPs when it
17-
# changed none. Before 2026-11-14 it ran the full pass every round, 10-45 min under load
18-
# and most of this step's wall time. The full pass is `--only mutation`, the flagless
19-
# run's mutation lane, and conformance.yml (tags, nightly, dispatch).
10+
# THE SPLIT. `lint` is `--only gates`, the one lane every change runs. `test` is
11+
# `--affected`: it lists what the branch changed against its merge base, maps the paths to
12+
# lanes with scripts/ci-lane-map.txt, prints which lanes it selected, skipped and why, and
13+
# runs only those (ts-fast + ts-unit for a TypeScript or web-client change; java-slow,
14+
# python or csharp for a change confined to that port; every selectable lane for a change
15+
# to fixtures, spec, scripts, workflows, agent-context, docs or any path the map does not
16+
# claim; none for a change to only the root README, CHANGELOG, CONTRIBUTING, AGENTS or CLAUDE
17+
# files, which the gates lane covers). Selected lanes run exactly as
18+
# `--only <lane> --no-integration` does, so a lane's contents are unchanged. Force every
19+
# lane with MO_CI_FULL=1. Before 2026-11-15 this step was always `--only ts-fast --only
20+
# ts-unit`, so a Java, Python or C# change was validated by the TypeScript tests and by
21+
# nothing in its own port. Measured green on a warm tree 2026-09-18: ts-fast + ts-unit
22+
# 256s, gates 23s. The test step's mutation gate is DIFF-SCOPED (gate_completeness_diff in
23+
# the script) and runs inside ts-fast: it mutates only the stryker.conf.mjs `mutate` files
24+
# the branch changed against its base, and SKIPs when it changed none. That is unchanged by
25+
# the selection. The full pass is `--only mutation`, the flagless run's mutation lane, and
26+
# conformance.yml (tags, nightly, dispatch).
2027
#
2128
# ORDER IS LOAD-BEARING, and it is why the work is split this way round rather than one
2229
# `--quick` in `test`. The pipeline runs test BEFORE lint. The `gates` section contains
@@ -39,18 +46,19 @@
3946
# NOT all of them, which is the part worth being exact about: five invoke `node` directly
4047
# (publish-set, script-name hooks, extract field kinds, metamodel-version, migration guides),
4148
# `check-publish-intent.sh` spawns `node` internally, and `gate_positioning_claims` invokes
42-
# bare `bun`. What holds across both kinds is the flag's safety: it cannot misfire on a
43-
# missing dotnet/mvn/uv, because nothing in `gates`, `ts-fast` or `ts-unit` invokes them.
49+
# bare `bun`. `gates`, `ts-fast` and `ts-unit` invoke no dotnet/mvn/uv. `--affected` can select the
50+
# lanes that do, and there the flag is the point: a change to that port with its toolchain
51+
# missing FAILS, where a plain run would skip it and report green.
4452
#
45-
# WHAT IS DELIBERATELY NOT HERE — the rest of the flagless `scripts/ci-local.sh`: the C#,
46-
# Java, Kotlin and Python conformance lanes, the full Java reactor, and
47-
# integration-tests.yml's five-port Testcontainers matrix plus the migrate-ts / runtime-ts
48-
# real-Postgres suites (`--only ts-slow`). Dropped for cost — they turn a ~4.5 minute step
49-
# into a much longer one on every round — and for dependence: they need dotnet, mvn, uv and
50-
# a live Docker daemon, which a gate host is not guaranteed to have, and neither
51-
# `--strict-toolchains` (a missing toolchain FAILS) nor its absence (a down Docker SKIPS
52-
# silently) gives an honest per-change signal when they are missing. docs/RELEASING.md
53-
# requires the flagless run before `bun publish`, which is where they are enforced.
53+
# WHAT IS DELIBERATELY NOT HERE — the docker half. `--affected` implies `--no-integration`:
54+
# integration-tests.yml's five-port Testcontainers matrix, the migrate-ts / runtime-ts
55+
# real-Postgres suites (`--only ts-slow`) and the cube live check need a live Docker daemon,
56+
# which a gate host is not guaranteed to have, and a down daemon SKIPS silently without
57+
# `--strict-toolchains`. They turn a ~4.5 minute step into a much longer one on every round.
58+
# A port's selected lane DOES need its toolchain (dotnet, mvn, uv), and under
59+
# `--strict-toolchains` a missing one FAILS rather than skipping, so a change to that port
60+
# is never reported green unchecked. docs/RELEASING.md requires the flagless run before
61+
# `bun publish`, which is where the docker half and every lane are enforced.
5462
#
5563
# NOTE: this file must be at the repo ROOT as `.no-mistakes.yaml` (NOT under
5664
# .no-mistakes/), and repo commands are read from the DEFAULT BRANCH — so nothing here
@@ -61,7 +69,7 @@
6169
# commands above already run the same script the GitHub workflows run.
6270

6371
commands:
64-
test: "scripts/ci-local.sh --only ts-fast --only ts-unit --strict-toolchains"
72+
test: "scripts/ci-local.sh --affected --strict-toolchains"
6573
lint: "scripts/ci-local.sh --only gates --strict-toolchains"
6674

6775
# Public repository: keep PR bodies short and the run intent out of them.
@@ -70,5 +78,37 @@ pr:
7078
appendix: collapsed
7179

7280
# Mark a failing test command as pre-existing when it also fails on the base.
81+
#
82+
# THE EVIDENCE PHASE. After `commands.test` passes, the gate ALWAYS runs an agent that derives
83+
# end-user scenarios and drives the product itself; no setting turns that turn off. It, not
84+
# `commands.test`, is the long part of this step: 264 recorded Test steps for this repository
85+
# average about 43 minutes, 130 of them over 30, against 4-5 for the command itself, and the
86+
# agent's budget is 75 minutes on this host. Its prompt is the one thing a repository controls,
87+
# so the runbook below scopes it the way `--affected` scopes the command: by the paths the
88+
# change touches (`path_instructions` match the changed-file list, as scripts/ci-lane-map.txt
89+
# does). This is guidance to a model, not a switch: it narrows what the agent is told to do
90+
# and does not bound it. The bound is the host's `test_agent_timeout`. These fields are read
91+
# from the default branch only, so they too take effect after merge.
7392
test:
7493
base_attribution: true
94+
instructions: |
95+
This repository ships libraries, a code generator and command line tools. There is no server or app to start. The test command already ran the lanes this change affects, and its output begins with the lanes it selected and skipped, so do not re-run the full suites or the lanes of ports the change does not touch. Drive the changed behavior directly instead: run the affected CLI command or generator on a small fixture from the fixtures directory, or the focused tests of the module the change touches. Keep to at most three scenarios and finish in about ten minutes. Report no-surface when the change has no runtime behavior, such as documentation, comments, CI configuration or metadata.
96+
path_instructions:
97+
- path: "docs/**"
98+
instructions: |
99+
Documentation only. There is nothing to start or drive. Return no-surface at once without running anything: the test command already ran the lanes whose tests read these pages, and the gates lane checks the prose.
100+
- path: "server/typescript/**"
101+
instructions: |
102+
TypeScript port. Drive the meta CLI from the workspace on a fixture, or run bun test in only the package the change touches. Do not run the Java, Python or C# suites.
103+
- path: "client/web/**"
104+
instructions: |
105+
Web client packages. Run bun test in only the package the change touches and check that its built entry still bundles. Do not run the server suites.
106+
- path: "server/java/**"
107+
instructions: |
108+
Java and Kotlin ports. Run mvn test for only the Maven modules the change touches, with -pl and -am, not the whole reactor. Do not run the TypeScript, Python or C# suites.
109+
- path: "server/python/**"
110+
instructions: |
111+
Python port. Run pytest on only the test modules the change touches. Do not run the TypeScript, Java or C# suites.
112+
- path: "server/csharp/**"
113+
instructions: |
114+
C# port. Run dotnet test on only the test project the change touches. Do not run the TypeScript, Java or Python suites.

‎AGENTS.md‎

Lines changed: 2 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -129,7 +129,8 @@ adds the C#/Java/Kotlin/Python conformance lanes, the Java reactor and `integrat
129129
Testcontainers matrix. Run `--quick` before opening a PR and the full script before a tag;
130130
`MO_CI_LIST_ONLY=1 scripts/ci-local.sh [flags]` prints what a selection would run without
131131
running it. The no-mistakes validation gate runs the script for you — see `.no-mistakes.yaml`,
132-
whose `lint` and `test` commands partition `--quick` between the two steps. That file is read
132+
whose `test` command runs `--affected` (only the lanes the change touches, per
133+
`scripts/ci-lane-map.txt`) and whose `lint` command runs `--only gates`. That file is read
133134
from the **default branch**, so it does nothing until it is merged to `main`.
134135

135136
**`leak-scan` is hosted again.** `main`'s protection requires that one status, and `hygiene.yml`

‎CONTRIBUTING.md‎

Lines changed: 31 additions & 10 deletions
Original file line numberDiff line numberDiff line change
@@ -94,11 +94,10 @@ definition of the checks: each wrapper checks out, installs the toolchains its
9494
lane needs, and calls the script, so a local run is the same check a workflow
9595
run would be. A pull request runs the leak scan (`hygiene.yml`) and `pr-ci.yml`, which
9696
runs the unit tests and conformance corpora of only the ports the PR touches (a change
97-
under `fixtures/`, `spec/`, `scripts/`, `.github/` or `agent-context/`, or to any file
97+
under `fixtures/`, `spec/`, `scripts/`, `.github/`, `agent-context/` or `docs/`, or to any file
9898
outside a port's directory that the selector does not recognise, touches every port; a
99-
change only to `docs/` or to Markdown files outside those directories runs just the
100-
offline gates). Its `pr-ci` job is green only when every selected lane is, and a newer
101-
push cancels the superseded run. The docker integration suites run on release tags and on
99+
change only to the root prose files above runs just the offline gates). Its `pr-ci` job is
100+
green only when every selected lane is, and a newer push cancels the superseded run. The docker integration suites run on release tags and on
102101
manual dispatch. Run the script locally before opening or merging a PR, so nothing red
103102
reaches `main`:
104103

@@ -110,9 +109,30 @@ scripts/ci-local.sh # full parity: hygiene.yml's leak-scan, all-port
110109
scripts/ci-local.sh --quick # hygiene.yml in full + the TypeScript half of
111110
# conformance.yml; skips the C#/Java/Kotlin/Python
112111
# ports, the java reactor and the docker suite
112+
scripts/ci-local.sh --affected # only the lanes this change needs (see below)
113113
MO_CI_LIST_ONLY=1 scripts/ci-local.sh --quick # print the steps, run nothing
114114
```
115115

116+
**`--affected` runs only what a change can break.** It lists the paths you changed since the
117+
merge base with the base branch (committed, uncommitted and untracked), maps each to lanes
118+
with the table in [`scripts/ci-lane-map.txt`](scripts/ci-lane-map.txt), and prints the lanes it
119+
selected, the lanes it skipped and the rule each path matched, so a reviewer can see what was
120+
not run. A change confined to one port runs that port (`ts-fast` and `ts-unit` for TypeScript
121+
and the web client, `java-slow`, `python`, `csharp`); a change to `fixtures/`, `spec/`,
122+
`scripts/` (the map included), `.github/`, `agent-context/`, anything under `docs/` (port
123+
tests read those pages) or any path no rule claims runs every lane, because unknown means
124+
full; a change to only the root `README.md`, `CHANGELOG.md`, `CONTRIBUTING.md`, `AGENTS.md`
125+
or `CLAUDE.md` runs no lane. It never runs `gates` (`--only gates` does, on every change) and
126+
never the docker half, so a change confined to the Java port skips the Java docker
127+
integration suites, which run on release tags and manual dispatch. When more than one port
128+
is selected their lanes run in parallel, as the full run's do, so a change is never slower
129+
than the full run; one port runs serially. Mutation testing is as before: diff-scoped inside
130+
`ts-fast`. `MO_CI_FULL=1 scripts/ci-local.sh --affected` forces every lane,
131+
`MO_CI_AFFECTED_BASE=<ref>` picks the base, and `scripts/ci-select-lanes.sh [<ref>]` explains
132+
a selection without running anything. The flagless run above is unchanged and is what
133+
[releases](docs/RELEASING.md) use. The same map drives `pr-ci.yml`'s port detection, and
134+
`scripts/test-ci-lane-selection.sh` (in the `gates` lane) pins what each kind of change selects.
135+
116136
The full run does its tree-writing steps first (install, the drift gates, one workspace
117137
build), then runs the rest as parallel lanes: at most `MO_CI_JOBS` at once, default 8. Each
118138
lane's output prints as one block when the lane finishes, so lanes never interleave. The
@@ -127,12 +147,13 @@ gate and a Java pom-version drift guard on every push (activate hooks once per
127147
clone: `git config core.hooksPath .githooks`).
128148

129149
If you drive changes through the no-mistakes validation gate, `.no-mistakes.yaml` pins its
130-
`test` step to `scripts/ci-local.sh --only ts-fast --only ts-unit --strict-toolchains` and
131-
its `lint` step to `scripts/ci-local.sh --only gates --strict-toolchains` — together exactly
132-
`--quick`, split so neither step repeats the other's work. `--strict-toolchains` is part of
133-
both: it promotes a SKIP to a FAIL, so a gate host missing bun cannot report green. Those
134-
commands are read from the **default branch**, so editing them on a feature branch has no
135-
effect until it merges.
150+
`test` step to `scripts/ci-local.sh --affected --strict-toolchains` and its `lint` step to
151+
`scripts/ci-local.sh --only gates --strict-toolchains`: the lanes the change affects, then
152+
the gates every change runs, split so the gates run after the build `test` does.
153+
`--strict-toolchains` is part of both: it promotes a SKIP to a FAIL, so a gate host missing
154+
bun (or, for a change to a port, that port's toolchain) cannot report green. Those commands are
155+
read from the **default branch**, so editing them on a feature branch has no effect until it
156+
merges.
136157

137158
## Releasing
138159

‎scripts/ci-affected-ports.sh‎

Lines changed: 20 additions & 21 deletions
Original file line numberDiff line numberDiff line change
@@ -3,11 +3,11 @@
33
#
44
# Usage: scripts/ci-affected-ports.sh <base-sha> <head-sha>
55
# Prints a space-separated subset of: ts java python csharp
6-
# (empty output = docs-only push: run only the cheap shared gates).
6+
# (empty output = a push touching only the root prose files: run just the cheap shared gates).
77
#
88
# Fail-open policy: an unknown base (force-push zeros), an unresolvable
9-
# range, or ANY path not matched by a rule selects ALL ports — this script
10-
# may only ever shrink coverage when it is certain.
9+
# range, or ANY path not matched by a rule in scripts/ci-lane-map.txt selects ALL
10+
# ports — this script may only ever shrink coverage when it is certain.
1111
#
1212
# NOT the whole selection. local-ci calls `ci-ports-to-run.sh`, which unions this
1313
# answer with the ports that are not known-green on main. "What changed" alone
@@ -17,25 +17,22 @@ set -uo pipefail
1717

1818
ALL="ts java python csharp"
1919

20+
# The path -> lane rules are scripts/ci-lane-map.txt, read through ci-select-lanes.sh, so
21+
# the no-mistakes gate (`ci-local.sh --affected`) and the hosted workflows share ONE table.
22+
# This script keeps its job of answering in PORTS, the unit the workflows' `detect` jobs
23+
# pass on, by mapping each selected lane to its port.
24+
# Docs are NOT prose-only in that table (port tests read docs/), so a docs/ change selects
25+
# every port here too; only the named root prose files select none.
26+
HERE="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
27+
MO_SELECT_LIB=1 source "$HERE/ci-select-lanes.sh"
28+
2029
map_paths_to_ports() { # args: changed paths; echoes the port set
21-
local p sel_ts=0 sel_java=0 sel_python=0 sel_csharp=0
22-
for p in "$@"; do
23-
case "$p" in
24-
server/typescript/*|client/web/*) sel_ts=1 ;;
25-
server/java/*) sel_java=1 ;;
26-
server/python/*) sel_python=1 ;;
27-
server/csharp/*) sel_csharp=1 ;;
28-
fixtures/*|spec/*|scripts/*|.github/*|agent-context/*)
29-
echo "$ALL"; return 0 ;;
30-
docs/*|*.md) : ;; # gates only
31-
*) echo "$ALL"; return 0 ;; # fail open
32-
esac
30+
select_lanes "$@"
31+
local lane port out=""
32+
for lane in $SELECTED_LANES; do
33+
port="$(lane_to_port "$lane")"
34+
case " $out " in *" $port "*) ;; *) out="$out $port" ;; esac
3335
done
34-
local out=""
35-
[ "$sel_ts" -eq 1 ] && out="$out ts"
36-
[ "$sel_java" -eq 1 ] && out="$out java"
37-
[ "$sel_python" -eq 1 ] && out="$out python"
38-
[ "$sel_csharp" -eq 1 ] && out="$out csharp"
3936
echo "${out# }"
4037
}
4138

@@ -47,7 +44,9 @@ if [ -z "$BASE" ] || [ "$BASE" = "0000000000000000000000000000000000000000" ] \
4744
|| ! git rev-parse -q --verify "$BASE^{commit}" >/dev/null 2>&1; then
4845
echo "$ALL"; exit 0
4946
fi
50-
CHANGED="$(git diff --name-only "$BASE" "$HEAD")" || { echo "$ALL"; exit 0; }
47+
# --no-renames: a rename's old path must be listed too, so moving a file OUT of a
48+
# port selects that port exactly like moving one in (matches ci-select-lanes.sh).
49+
CHANGED="$(git diff --no-renames --name-only "$BASE" "$HEAD")" || { echo "$ALL"; exit 0; }
5150
[ -z "$CHANGED" ] && { echo ""; exit 0; }
5251
# shellcheck disable=SC2086
5352
mapfile -t paths <<<"$CHANGED"

0 commit comments

Comments
 (0)