Skip to content

ci: run only the lanes a change affects in local CI and the validation gate - #589

Merged
dmealing merged 5 commits into
mainfrom
fm/mo-ci-affected-lanes
Nov 15, 2026
Merged

dmealing merged 5 commits into
mainfrom
fm/mo-ci-affected-lanes

Conversation

@dmealing

@dmealing dmealing commented Nov 15, 2026 •

Copy link
Copy Markdown
Member

What Changed

  • Added scripts/ci-local.sh --affected: it lists the paths changed since the merge base with the base branch (committed, uncommitted and untracked), maps them to lanes with one table, scripts/ci-lane-map.txt (read by the new scripts/ci-select-lanes.sh), prints the lanes it selected, the lanes it skipped and the rule each path matched, and runs only the selected lanes. When more than one port is selected their lanes run in parallel, like the full run's, so a shared change is not serialized. Mutation stays diff-scoped inside ts-fast. MO_CI_FULL=1 forces every lane, MO_CI_AFFECTED_BASE=<ref> picks the base, and the flagless run is unchanged and is what releases use.
  • The map: a path under one port runs that port (ts-fast + ts-unit for TypeScript and the web client, java-slow, python, csharp); fixtures/, spec/, scripts/ (this map included), .github/, agent-context/ and everything under docs/ run every lane, because port tests read those files; only the root README.md, CHANGELOG.md, CONTRIBUTING.md, AGENTS.md and CLAUDE.md run none. Any path no rule claims, an unreadable or invalid map, an unresolvable base and a git failure all select every lane (unknown means full).
  • The same map now drives scripts/ci-affected-ports.sh (port detection for pr-ci.yml and local-ci.yml), with --no-renames so a rename's old path still selects the port it left, and the gate and the hosted workflows share one base-ref rule.
  • .no-mistakes.yaml: test is now scripts/ci-local.sh --affected --strict-toolchains (it was --only ts-fast --only ts-unit, so a Java, Python or C# change was validated by TypeScript tests only); lint stays --only gates. A test: block scopes the evidence agent by changed path.
  • Tests: scripts/test-ci-lane-selection.sh (in the gates lane) pins what each kind of change selects, the fail-open cases, a throwaway-git end to end, the parallel plan, and rename handling. Docs: CONTRIBUTING.md, AGENTS.md, .no-mistakes.yaml, the pr-ci.yml header comment.

Wall times

scripts/ci-local.sh on one warm tree, one run each, on a busy shared host (load average 15 to 50 on 32 threads), so read them as rough. Before is the old Test command, --only ts-fast --only ts-unit, which ran for every change. After is --affected --strict-toolchains.

Change Before After Lanes run after
TypeScript only 851s 503s ts-fast, ts-unit
Java only 851s 386s java-slow
Python only 851s 194s python
C# only 851s 148s csharp
Docs-only (root prose files) 851s 0s none (the gates lane still runs in Lint)
Shared (fixtures/, scripts/, ...) 851s 325s all four ports in parallel, covering more than before

No change type got slower, and a shared change now covers every port in less wall time than the old TypeScript-only command took.

Caveats

  1. The gate change does not apply to this PR's own run. .no-mistakes.yaml is read from the default branch, so this run's Test step still executed the old command (4m00s); --affected and the new test: block take effect after merge. The times above are local measurements of the new command.
  2. The evidence agent is not switched off. After the Test command passes, the gate always runs an agent that derives and drives end-user scenarios, and no setting turns that off; for this run the Test step took 23m35s, almost all of it that agent, against 4 to 5 minutes for the command. This PR scopes it only by prompt (trusted test.instructions and test.path_instructions, which also take effect after merge). That narrows what it is told to do and does not bound it. A hard bound needs a lower host test_agent_timeout, or a no-mistakes feature to skip the evidence agent for given paths, roughly one to two days of work in that repository.
  3. Wider than before in two places. Everything under docs/ now runs every lane in the hosted workflows as well, where it used to run none, because Java, TypeScript and Python tests read those pages. --affected never runs the docker half, so a change confined to one port skips that port's docker integration suites, which still run on release tags and on manual dispatch.
  4. Only the lanes' selection is gated by tests. The timings are single noisy runs, not benchmarks, and the selection tests assert which lanes run, not how long they take.

Pipeline

One no-mistakes run: the review step raised two findings, both fixed in that run.

Add scripts/ci-lane-map.txt, one path-to-lane table, and scripts/ci-select-lanes.sh,
which reads it. ci-local.sh --affected lists the change since the merge base, prints the
lanes it selected and skipped and the rule each path matched, and runs the selected ones;
several ports run as parallel lanes like the full run, one stays serial. Unknown paths, an
unreadable map and an unresolvable base all select every lane. ci-affected-ports.sh reads
the same map, so the gate and the hosted workflows agree. docs/recipes is executed by port
tests and no longer counts as docs.

.no-mistakes.yaml: the test command is --affected, and test.instructions and
test.path_instructions scope the evidence agent by the paths a change touches.
…e lane map once

Review follow-ups for affected-lane selection: docs/ maps to every lane because port
tests read those pages (only the root prose files select none), the selector CLI uses the
same base rule as the gate, the map is parsed once per selection without per-line forks,
and the selected-port lanes call the shared step functions directly.
@dmealing
dmealing merged commit 624a54c into main Nov 15, 2026
9 checks passed
@dmealing
dmealing deleted the fm/mo-ci-affected-lanes branch November 15, 2026 14:53
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant