ci: run mutation testing post-merge as its own lane

Summary

PLAN Phase 13 and #7581 decide that mutation testing runs post-merge only. /mutation-testing:audit is a model-driven skill (plugins/mutation-testing/README.md:11, :33-39), and nothing on main runs it (no .claude/mutation-testing.md outside eval fixtures, no workflow). Owner decision Q675 (2026-10-22): split this out of #9184 into its own slice.

Fix

Open questions for the build's interview: which merges trigger a run (every merge to main, or a schedule over the day's merges), the scope per run (the merged diff's changed source files), the token and permissions under ADR 0049's hardening, and how a surviving mutant is reported (an issue per run, or a comment on the merged PR). It is a new Claude lane, so it needs Automation Lanes' design and the Security & Cost session's review.

Attended build (not lane): .github/** change, predicted risk HIGH.

Verification

A post-merge run on a merge that changes a source file reports its killed and surviving mutants.
No pull request run starts it.

Decision recorded. The owner answered in the Issue Triage session (2026-10-25), deciding the open items on the facts and research:

go with whatever the facts/data/metrics + /discovery:research says and ensure we're aligned with SOLID, DRY/SSOT principles

[quote truncated; the full decision is at the link]

#9195 (comment)

That decision settles the open questions: trigger, token, report location and spend. A later comment refines the report job. The model job stays read-only, and a model-free job posts the PR comment. Nothing is left to ask. The build is attended, because it edits .github/**. I found no duplicates.

_{Automated triage, applied by the lanes App.}

Released for re-triage under the owner's 2026-10-24 rule: an issue stops for a person only for an intent or vision question, a security call, a host or attended run, or spend outside prepaid credit. The hold label came off and intake triage runs again now; it re-applies a hold, with a numbered question, when one of those applies.

To hold this issue again, add needs-human (or status: needs-decision) and say why in a comment.

The latest triage verdict holds nothing: the issue is at status: ready.

Refinement. A read-only run explored this repository, searched its issues and pull requests, and read the allowed docs to draft what follows. On an issue a listed human filed, it counts as the issue's spec once posted, while unedited; it is still model text, so check it.

Intended outcome: A new post-merge-only workflow under .github/workflows runs /mutation-testing:audit on the merged diff's source files and reports killed and surviving mutants, never starting on a pull request.

Dependencies

#9184: this issue was split out of it; confirm its remaining scope does not overlap
ADR 0049 hardening and the Automation Lanes design and Security & Cost review, which the issue says a new Claude lane needs

Related work

pr-merge: GITHUB_TOKEN-dispatched post-merge runs file no demotion issue #7528 (related): Post-merge runs dispatched by GITHUB_TOKEN file no demotion issue; may affect how this lane is triggered.

Open questions

Which merges trigger a run: every merge to main touching source files, or a schedule over the day's merges?
RECOMMENDED: Every push to main whose diff touches source files, using a push trigger with a path filter and no pull_request trigger. Basis: judgment
Which token and permissions does the lane run with under ADR 0049 hardening, and who approves them? This is a security call that needs the Security & Cost review.
RECOMMENDED: contents: read plus the narrowest write scope the chosen reporting channel needs (pull-requests: write for a PR comment, or issues: write for an issue), with no secrets exposed to mutated code. Basis: judgment
How is a surviving mutant reported: an issue per run, or a comment on the merged PR?
RECOMMENDED: A comment on the merged PR, which keeps results next to the change and avoids issue noise. A direct push to main has no PR, so fall back to the job summary. Basis: judgment
What runs the audit unattended? The audit is a model-driven skill, no workflow runs it today, and there is no .claude/mutation-testing.md config outside eval fixtures. Does the lane commit a config, use the no-config mode of #8766, or run setup first?
RECOMMENDED: Commit a repo config via setup before the lane lands, since audit runs deterministically from the tracked config. Inference spend must be confirmed as within prepaid credit. Basis: plugins/mutation-testing/README.md:10
Is this an attended build only? The issue says it is an attended .github/** change with predicted risk HIGH, and its Verification section has no checkable test form.
RECOMMENDED: Yes, keep it attended. Add a task list once the four design answers are decided, so a lane can verify it. Basis: judgment

Sources read

plugins/mutation-testing/README.md:11: audit is diff-scoped, read-only on tracked source, with at most one mutant per changed line, reverting every mutant
plugins/mutation-testing/README.md:39: states no whole-repo nightly ratchet, so the lane must stay diff-scoped
.github/workflows: no workflow mentions mutation testing; post-merge-demotion-alert.yml and post-merge-detect-silent-revert.yml are existing post-merge patterns to follow

_{Automated refinement by the lanes App. Answer with a comment whose first line is **Decision recorded.**.}

Decision recorded (Automation Lanes, same rule). Criterion 3's literal text can't be met by any model-driven audit, because mutants run inside the model step. The control is CLAUDE_CODE_SUBPROCESS_ENV_SCRUB, which keeps the OAuth token out of Bash children (pr-run-activity-write.yml:1396-1403), and the mutated code is main's post-merge code, not PR code. Decision 3 (the comment on the merged PR) is built as a model-free report job that writes with its own GITHUB_TOKEN, on the precedent of maintenance-audit-lane-runs.yml:269-279. The model job stays read-only.

The work-resolve-issue lane stopped on this issue (patch-protected-path): the change needs a path the issue lane may not push, which a maintainer session builds by hand.
It added host-run and removed agent-ready: an agent run on a host builds it, and it waits on no decision from a person.

In the run's words:

The run stopped early (patch-protected-path): its change needs a path the issue lane may not push, which a person changes instead. In the run's words: patch-protected-path: .github/workflows/** (the issue's change is a new post-merge workflow under .github/workflows/ that runs /mutation-testing:audit on push to main and reports via a model-free PR-comment job; the lane cannot push that path, and the issue's 2026-10-26 builder comment assigns the build to the Automation Lanes session from the owner's machine)

What the run says it did not do:

The whole issue: a new .github/workflows/ post-merge mutation-testing workflow, a protected path for this lane; left to the attended Automation Lanes build
Repository checks (check-changelog-fragments, check-affected), tidy, review and verify were not run: the run stopped before any edit, so the working tree is unchanged

Run: https://github.com/melodic-software/claude-code-plugins/actions/runs/40814302234

To hand it back to the lane, remove host-run and add agent-ready again.

Closing as done by #10286, which built this lane (post-merge-select-mutation-range.yml and post-merge-test-mutation.yml) to the decisions recorded on this issue: one run per push to main that touches mutate paths, reported as a comment on the merged pull request. #10587 and #10742 followed it.

Verification against the done-when list:

A post-merge run on a merge that changes a source file reports its killed and surviving mutants: run https://github.com/melodic-software/claude-code-plugins/actions/runs/40810523674 returned outcome: reported, 2 killed and 5 survived in plugins/session-flow/skills/audit-sessions/scripts/sweep.py, and its report job ran the merged-PR comment step.
No pull request run starts it: its only triggers are push to main (range selector) and the workflow_run that follows.
No secret reaches the step that runs mutants: the mutants run inside the model step, whose job holds contents: read only and runs under CLAUDE_CODE_SUBPROCESS_ENV_SCRUB with --setting-sources user and a tool allowlist (the control Automation Lanes recorded here on 2026-10-26). The write job holds pull-requests: write and runs no model.

The host-run label was added at 03:53Z by the issue lane, which did not see that #10286 had already built the workflow.

D22 proof for PR #10286 (#7555 criterion 3).

Check: the model-free "Check the checkout is restored" step in .github/workflows/post-merge-test-mutation.yml (job audit). It fails the job when tracked files differ from the pushed head after the mutation audit. Slice: #9195. PR: #10286

Commits

Merge: 67f071560742b58ce31a17920948c4770d2a108e
Parent (67f0715607^1): 75d7a773f185f1eef8dbc59e996e81a165045c1f. It has no post-merge-test-mutation.yml (ls .github/workflows | grep -i mutat prints nothing), so the step did not exist before this PR.

Commands. The step's run: block was taken verbatim from the merge commit with yq '.jobs.audit.steps[] | select(.name == "Check the checkout is restored") | .run' .github/workflows/post-merge-test-mutation.yml, saved as restored-step.sh, and run in a detached worktree of the merge commit:

# 1. clean tree
env -C <merge> GITHUB_OUTPUT=gh-output-clean.txt GIT_CONFIG_GLOBAL=/dev/null GIT_CONFIG_NOSYSTEM=1 bash restored-step.sh
# 2. planted: one line added to a tracked file (scripts/run-covering-tests.sh), as an unreverted mutant would leave it
env -C <merge> GITHUB_OUTPUT=gh-output-planted.txt ... bash restored-step.sh
# 3. reverted
git -C <merge> restore --source=HEAD -- scripts/run-covering-tests.sh
env -C <merge> GITHUB_OUTPUT=gh-output-reverted.txt ... bash restored-step.sh

Red (planted modification, exit 1)

 scripts/run-covering-tests.sh | 1 +
 1 file changed, 1 insertion(+)
::error::the checkout differs from the pushed head after the audit
exit=1
restored=false

Green (clean tree and after revert, exit 0)

exit=0
restored=true

The run after revert printed the same: exit=0, restored=true.

Scope note. The step compares tracked files only (git diff --quiet HEAD; the workflow comment says "Tracked files must equal the pushed head"). An untracked file left by the audit is not caught, which matches its stated scope.

Verdict: PROVEN as a planted violation that fails and passes after revert (PLAN.md:42-43). This is the step's own script run locally, not a GitHub Actions run. There is no parent-red run because the parent has no such step to run.

Navigation Menu

Repository navigation

ci: run mutation testing post-merge as its own lane #9195

Summary

Fix

Verification

Related

7 remaining items

Metadata

Assignees

Labels

Type

Fields

Projects

Milestone

Relationships

Development

ci: run mutation testing post-merge as its own lane #9195

Description

Summary

Fix

Verification

Related

Activity

github-actions commented on Oct 22, 2026

kyle-sexton commented on Oct 25, 2026

melodic-automation-lanes commented on Oct 25, 2026

melodic-automation-lanes commented on Oct 25, 2026

7 remaining items

kyle-sexton commented on Oct 26, 2026

melodic-automation-lanes commented on Oct 29, 2026

kyle-sexton commented on Oct 29, 2026

kyle-sexton commented on Nov 7, 2026

Metadata

Metadata

Assignees

Labels

Type

Fields

Projects

Milestone

Relationships

Development

Issue actions