Skip to content

Add mutation-testing reusable workflows for cargo-mutants and mutmut#319

Merged
leynos merged 15 commits into
mainfrom
add-mutation-testing-workflows
Jul 5, 2026
Merged

Add mutation-testing reusable workflows for cargo-mutants and mutmut#319
leynos merged 15 commits into
mainfrom
add-mutation-testing-workflows

Conversation

@leynos

@leynos leynos commented Jul 4, 2026

Copy link
Copy Markdown
Owner

Summary

This branch implements scheduled, informational mutation-testing reusable workflows for the leynos estate: mutation-cargo.yml for Rust repositories using cargo-mutants, and mutation-mutmut.yml for uv-managed Python repositories using mutmut. Both generalize the workflow proven in leynos/wireframe (its ADR-007): a daily-cron-friendly change-detection guard makes quiet days a cheap no-op, scoped runs mutate only recently changed files, full dispatch runs fan out across shards (cargo only), and surviving mutants land in the job summary and artefacts. Neither workflow gates pull requests.

Execplan: docs/execplans/add-mutation-testing-workflows.md — Stages A–F implemented in this branch; Stage G (pilot caller validation on wireframe and cmd-mox) is deliberately deferred until merge, because callers must pin a shared-actions SHA.

Key design points, each recorded in the execplan's Decision Log:

  • Non-trivial logic lives in tested Python helper scripts under workflow_scripts/ (per the scripting standards), not in workflow shell; 47 unit tests cover detection bucketing, window edges, shard matrices, argument construction, exit-code mapping, report merging, results parsing, and summary rendering.
  • The cargo exit-code contract treats cargo-mutants exit codes 2 (missed mutants) and 3 (timeouts) as informative successes while 1/4/70 fail the job; mutmut needs no masking (it exits 0 with survivors; non-zero means a failing baseline).
  • Full cargo dispatch runs shard via a detect-emitted matrix (shard-count, default 6) with a merge-and-summarize job — sized from wireframe's measured first run (1,821 mutants ≈ 14–15 single-job hours, beyond GitHub's 360-minute ceiling). mutmut has no shard equivalent, so its workflow is a single job.
  • An empirical mutmut 3.6 spike corrected stale assumptions before implementation: source_paths replaces paths_to_mutate, file paths are rejected as run arguments (changed files translate to module globs such as pkg.mod.*), and results are parsed from mutmut results --all true.
  • Tool version pins (cargo-mutants 27.1.0, mutmut 3.6.0) default in the workflows because both report formats are unstable; the parsers and pins travel together.
  • Each job reuses the dependabot-automerge OIDC source-resolution and ACT bypass pattern, keeping helper scripts in lockstep with the caller's pinned SHA.

Review walkthrough

Validation

  • make check-fmt: pass (207 files already formatted)
  • make typecheck: pass (ty check, both search-path groups)
  • make lint: pass (ruff + action-validator over all action manifests)
  • make test: pass — 989 passed, 14 skipped
  • make markdownlint: pass (69 files, 0 errors)
  • make nixie: pass (all Mermaid diagrams)
  • coderabbit review --agent: completed, 0 findings
  • act guard-path integration tests: not run locally (act is not installed on the implementation machine); they are opt-in via ACT_WORKFLOW_TESTS=1 make test and should be confirmed on a machine with act before or during Stage G.

Notes

  • The act tests cover the change-detection skip path end-to-end; the mutation-run path cannot be act-tested because stub binaries cannot be injected onto a workflow_call job's PATH — it is covered by the unit tests and by Stage G pilot runs (see the Decision Log entry revising the original stubbed-tools intent).
  • mutation-mutmut.yml requires callers to be uv-managed projects with [tool.mutmut] configured; pip-only projects are out of scope for the first release.
  • Follow-up after merge (Stage G): migrate wireframe from its bespoke workflow to a mutation-cargo.yml caller, add a mutmut caller to cmd-mox, verify real reports parse, and adjust version pins if the pilots surface drift.

@sourcery-ai sourcery-ai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sorry @leynos, you have reached your weekly rate limit of 500000 diff characters.

Please try again later or upgrade to continue using Sourcery

@coderabbitai

coderabbitai Bot commented Jul 4, 2026

Copy link
Copy Markdown
Contributor

Review Change Stack

Note

Reviews paused

It looks like this branch is under active development. To avoid overwhelming you with review comments due to an influx of new commits, CodeRabbit has automatically paused this review. You can configure this behavior by changing the reviews.auto_review.auto_pause_after_reviewed_commits setting.

Use the following commands to manage reviews:

  • @coderabbitai resume to resume automatic reviews.
  • @coderabbitai review to trigger a single review.

Use the checkboxes below for quick actions:

  • ▶️ Resume reviews
  • 🔍 Trigger review

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info
⚙️ Run configuration

Configuration used: Organization UI

Review profile: ASSERTIVE

Plan: Pro Plus

Run ID: 2364813c-c10d-4a38-be10-4b6aee55829b

📥 Commits

Reviewing files that changed from the base of the PR and between 3d02f39 and ee10d5e.

📒 Files selected for processing (13)
  • .github/actions/resolve-workflow-source/README.md
  • .github/actions/resolve-workflow-source/action.yml
  • .github/workflows/mutation-cargo.yml
  • .github/workflows/mutation-mutmut.yml
  • .github/workflows/test-resolve-workflow-source.yml
  • README.md
  • docs/developers-guide.md
  • docs/execplans/add-mutation-testing-workflows.md
  • tests/workflows/test_resolve_workflow_source.py
  • workflow_scripts/mutation_detect_changes.py
  • workflow_scripts/mutation_run_mutmut.py
  • workflow_scripts/tests/test_mutation_run_mutmut.py
  • workflow_scripts/tests/test_mutation_summarize_cargo.py
🔗 Linked repositories identified

CodeRabbit considers these linked repositories for cross-repo context during reviews:

  • leynos/polythene (auto-detected)

Adds two reusable, informational mutation-testing workflows: one for Rust repositories using cargo-mutants, and one for uv-managed Python repositories using mutmut. Both workflows use shared change detection to skip quiet days, run only on recently changed files for scheduled runs, and publish results via job summaries and artefacts without blocking pull requests.

Also adds the supporting workflow scripts for detecting changes, running and summarising cargo-mutants, and running/parsing/summarising mutmut, alongside unit tests, property tests, act-based integration coverage for the skip path, and documentation for callers and maintainers. The new execplan docs/execplans/add-mutation-testing-workflows.md captures the rollout and design decisions behind the implementation.

Includes a reusable Resolve workflow source composite action and updates the workflows and docs to use it, plus small test fixture updates for scheduled and manual workflow events.

Walkthrough

Add reusable mutation-testing workflows for Rust and Python, plus the helper scripts, tests, fixtures, and docs that wire change detection, execution, summary reporting, and workflow-source resolution end to end.

Changes

Mutation testing workflows

Layer / File(s) Summary
Change detection and matrix building
workflow_scripts/mutation_detect_changes.py, workflow_scripts/tests/test_mutation_detect_changes.py, workflow_scripts/tests/test_mutation_properties.py, tests/workflows/fixtures/*.json
DetectionConfig and MatrixEntry define change-detection inputs and matrix entries; git-scanned file sets are bucketed, sharded, serialised, and covered by unit, property, and act-fixture tests.
cargo-mutants run script and tests
workflow_scripts/mutation_run_cargo.py, workflow_scripts/tests/test_mutation_run_cargo.py
MutantsInvocation, build_arguments, and interpret_exit_code build cargo-mutants invocations and classify informative exit codes, with CLI coverage for argument assembly and invalid sharding.
cargo-mutants summary aggregation and tests
workflow_scripts/mutation_summarize_cargo.py, workflow_scripts/tests/test_mutation_summarize_cargo.py, workflow_scripts/tests/test_mutation_properties.py
parse_outcomes, collect_reports, and render_summary merge shard artefacts into target reports and Markdown summaries, with CLI and property coverage for report merging and output emission.
mutmut run script and tests
workflow_scripts/mutation_run_mutmut.py, workflow_scripts/tests/test_mutation_run_mutmut.py, workflow_scripts/tests/test_mutation_properties.py
files_to_module_globs, parse_results, count_statuses, and render_summary translate changed Python files into mutmut runs and summaries, with CLI and property coverage.
Reusable workflows, resolve action, and act harness
.github/actions/resolve-workflow-source/*, .github/workflows/mutation-cargo.yml, .github/workflows/mutation-mutmut.yml, .github/workflows/test-mutation-cargo.yml, .github/workflows/test-mutation-mutmut.yml, .github/workflows/test-resolve-workflow-source.yml, tests/workflows/*
Adds reusable workflow-call entrypoints, OIDC-based workflow-source resolution, skip-path caller workflows, and act-based integration tests with fixtures for scheduled-event behaviour and resolve-action branches.
Documentation and execplan
README.md, docs/developers-guide.md, docs/execplans/add-mutation-testing-workflows.md, docs/mutation-cargo-workflow.md, docs/mutation-mutmut-workflow.md, .github/actions/resolve-workflow-source/README.md
Adds the workflow index entry, maintainer guide, execplan, per-workflow usage docs, and action documentation for the reusable mutation-testing workflows and workflow-source resolver.

Suggested labels: Roadmap

Poem

Detect the drift, then shard and sing,
Rust and Python take their spring.
Reports combine in tidy rows,
While OIDC quietly knows.
~~~ the mutants march, the summaries glow ~~~

🚥 Pre-merge checks | ✅ 18 | ❌ 2

❌ Failed checks (2 warnings)

Check name Status Explanation Resolution
Title check ⚠️ Warning The title matches the new mutation-testing workflows, but it omits the required execplan reference for this roadmapped change. Add the execplan reference to the title, for example: 'Add mutation-testing reusable workflows for cargo-mutants and mutmut (add-mutation-testing-workflows)'.
User-Facing Documentation ⚠️ Warning docs/users-guide.md is absent; the new mutation workflows are only signposted in README and workflow-specific docs, not in the required user guide. Add a users guide page covering the new mutation-testing workflows, their purpose, and getting-started links, then keep any translated guides in sync.
✅ Passed checks (18 passed)
Check name Status Explanation
Description check ✅ Passed The description clearly matches the mutation-testing workflow and helper-script changes in this pull request.
Docstring Coverage ✅ Passed No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check.
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.
Testing (Overall) ✅ Passed Accept it: direct unit, property, and act integration tests exercise detection, scoping, summaries, exit-code paths, and workflow skip wiring.
Developer Documentation ✅ Passed The developer guide now documents the new workflows, helper boundaries, pins, and act/testing notes; the execplan records decisions and shows Stages A–F checked off with Stage G outstanding.
Module-Level Documentation ✅ Passed PASS: Every touched Python module has a module docstring; the helper scripts explain purpose and workflow links, and the test modules state their role against those components.
Testing (Unit And Behavioural) ✅ Passed PASS: Unit tests cover edge/error/invariant cases for detection, runners, and summary; act-based integration tests hit the real workflow boundary and skip path end-to-end.
Testing (Property / Proof) ✅ Passed PASS: Hypothesis covers the new invariants—CSV splitting, bucketing, glob translation, result parsing, outcome counting, and scoped matrix path stripping.
Testing (Compile-Time / Ui) ✅ Passed No compile-time Rust surface changed, so trybuild is not applicable; Markdown/CLI outputs are covered by focused assertions rather than snapshots.
Unit Architecture ✅ Passed PASS: query helpers are pure and isolated, while side-effects stay in small app entrypoints; the refactors add explicit config/result dataclasses and boundary tests.
Domain Architecture ✅ Passed Changes stay in workflow adapters/tests/docs; no core domain code is touched, and the scripts translate env/CLI/filesystem concerns at the boundary.
Observability ✅ Passed Structured key=value emits cover detect/run/summarise decision points, failures log explicit error messages, and job summaries/artifacts expose skip and report state.
Security And Privacy ✅ Passed PASS: Permissions stay minimal, actions are pinned, no secrets or write-all scopes appear, and subprocess calls avoid shell evaluation.
Performance And Resource Use ✅ Passed All new loops are linear and bounded by shard count, changed files, or artefact dirs; each file/report is read once, with no unbounded retries or recursive walks.
Concurrency And State ✅ Passed PASS: Parallel cargo shards are isolated per job, fail-fast is false, artefacts are uniquely named, and merge/order behaviour is unit-tested; no shared mutable state or async tasks appear.
Architectural Complexity And Maintainability ✅ Passed Every new abstraction has an immediate seam and reuse path: the OIDC resolver is shared by multiple workflows, and the helper dataclasses/predicates simplify tests and signatures.
Rust Compiler Lint Integrity ✅ Passed PR touches workflows, docs, and Python helpers only; no .rs/Cargo files or Rust lint suppressions were introduced, so the Rust-specific lint check is not applicable.
✨ Finishing Touches
📝 Generate docstrings
  • Create stacked PR
  • Commit on current branch
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Commit unit tests in branch add-mutation-testing-workflows

Comment @coderabbitai help to get the list of available commands.

codescene-access[bot]

This comment was marked as outdated.

codescene-access[bot]

This comment was marked as outdated.

codescene-access[bot]

This comment was marked as outdated.

codescene-access[bot]

This comment was marked as outdated.

codescene-access[bot]

This comment was marked as outdated.

codescene-access[bot]

This comment was marked as outdated.

@leynos leynos changed the title Create mutation-testing reusable workflows Add mutation-testing reusable workflows for cargo-mutants and mutmut Jul 4, 2026
codescene-access[bot]

This comment was marked as outdated.

codescene-access[bot]

This comment was marked as outdated.

@leynos

leynos commented Jul 4, 2026

Copy link
Copy Markdown
Owner Author

@coderabbitai Please suggest a fix for this issue and supply a prompt for an AI coding agent to enable it to apply the fix. Include the file and symbol names indicated in the issue at the head of your response.

Excess Number of Function Arguments

workflow_scripts/mutation_run_cargo.py: build_arguments

What lead to degradation?

build_arguments has 7 arguments, max arguments = 4

Why does this problem occur?

Functions with many arguments indicate either a) low cohesion where the function has too many responsibilities, or b) a missing abstraction that encapsulates those arguments.
The threshold for the Python language is 4 function arguments.

How to fix it?

Start by investigating the responsibilities of the function. Make sure it doesn't do too many things, in which case it should be split into smaller and more cohesive functions. Consider the refactoring INTRODUCE PARAMETER OBJECT to encapsulate arguments that refer to the same logical concept.

@leynos

leynos commented Jul 4, 2026

Copy link
Copy Markdown
Owner Author

@coderabbitai Please suggest a fix for this issue and supply a prompt for an AI coding agent to enable it to apply the fix. Include the file and symbol names indicated in the issue at the head of your response.

Complex Conditional

workflow_scripts/mutation_run_mutmut.py: parse_results

What lead to degradation?

parse_results has 1 complex conditionals with 2 branches, threshold = 2

Why does this problem occur?

A complex conditional is an expression inside a branch such as an if-statmeent which consists of multiple, logical operations. Example: if (x.started() && y.running()).Complex conditionals make the code even harder to read, and contribute to the Complex Method code smell. Encapsulate them.

How to fix it?

Apply the DECOMPOSE CONDITIONAL refactoring so that the complex conditional is encapsulated in a separate function with a good name that captures the business rule. Optionally, for simple expressions, introduce a new variable which holds the result of the complex conditional.

Helpful refactoring examples

To get a general understanding of what this code health issue looks like - and how it might be addressed - we have prepared some diffs for illustrative purposes.

SAMPLE

# complex_conditional.js
 function messageReceived(message, timeReceived) {
-   // Ignore all messages which aren't from known customers:
-   if (!message.sender &&
-       customers.getId(message.name) == null) {
+   // Refactoring #1: encapsulate the business rule in a
+   // function. A clear name replaces the need for the comment:
+   if (!knownCustomer(message)) {
      log('spam received -- ignoring');
      return;
    }
 
-  // Provide an auto-reply when outside business hours:
-  if ((timeReceived.getHours() > 17) ||
-      (timeReceived.getHours() < 8)) {
+  // Refactoring #2: encapsulate the business rule.
+  // Again, note how a clear function name replaces the
+  // need for a code comment:
+  if (outsideBusinessHours(timeReceived)) {
     return autoReplyTo(message);
   }
 
   pingAgentFor(message);
+}
+
+function outsideBusinessHours(timeReceived) {
+  // Refactoring #3: replace magic numbers with
+  // symbols that communicate with the code reader:
+  const closingHour = 17;
+  const openingHour = 8;
+
+  const hours = timeReceived.getHours();
+
+  // Refactoring #4: simple conditional rules can
+  // be further clarified by introducing a variable:
+  const afterClosing = hours > closingHour;
+  const beforeOpening = hours < openingHour;
+
+  // Yeah -- look how clear the business rule is now!
+  return afterClosing || beforeOpening;
 }

@coderabbitai

This comment was marked as resolved.

@coderabbitai

coderabbitai Bot commented Jul 4, 2026

Copy link
Copy Markdown
Contributor

Tip

For best results, initiate chat on the files or code changes.

🧩 Analysis chain

🏁 Script executed:

#!/bin/bash
fd mutation_run_mutmut.py
echo "---"
ast-grep run --pattern 'def parse_results($$$) {
  $$$
}' --lang python workflow_scripts/mutation_run_mutmut.py
echo "---full file---"
cat -n workflow_scripts/mutation_run_mutmut.py

Length of output: 2194


🏁 Script executed:

#!/bin/bash
fd mutation_run_mutmut.py
echo "---"
cat -n workflow_scripts/mutation_run_mutmut.py

Length of output: 10841


File: workflow_scripts/mutation_run_mutmut.py
Symbol: parse_results

Extract the multi-clause guard into a named predicate so the business rule ("this line represents a mutant result") is legible without unpacking boolean algebra.

 def parse_results(text: str) -> list[MutantResult]:
     """Parse ``mutmut results --all true`` output into mutant results.
 
     Parameters
     ----------
     text : str
         Raw command output; result lines have the form
         ``    name: status``.
 
     Returns
     -------
     list[MutantResult]
         Parsed results, in output order. Lines that do not look like
         mutant results (warnings, blank lines) are ignored.
     """
     results: list[MutantResult] = []
     for raw_line in text.splitlines():
         line = raw_line.strip()
         name, separator, status = line.partition(": ")
-        if not separator or " " in name or "__mutmut_" not in name:
-            continue
+        if not _is_mutant_result_line(name, separator):
+            continue
         results.append(MutantResult(name=name, status=status.strip()))
     return results
+
+
+def _is_mutant_result_line(name: str, separator: str) -> bool:
+    """Decide whether a parsed ``name``/``separator`` pair is a mutant result.
+
+    Parameters
+    ----------
+    name : str
+        Text preceding the ``: `` separator, expected to be a mutant name.
+    separator : str
+        The partition separator; empty when ``": "`` was absent from the line.
+
+    Returns
+    -------
+    bool
+        ``True`` when the line looks like a ``name: status`` mutant result
+        rather than a warning or blank line.
+    """
+    has_separator = bool(separator)
+    is_single_token = " " not in name
+    is_mutant_name = "__mutmut_" in name
+    return has_separator and is_single_token and is_mutant_name

Prompt for an AI coding agent:

In `workflow_scripts/mutation_run_mutmut.py`, decompose the complex conditional in `parse_results`.

Extract the guard `if not separator or " " in name or "__mutmut_" not in name:` into a new module-level helper function `_is_mutant_result_line(name: str, separator: str) -> bool` placed immediately after `parse_results`. Give it a NumPy-style docstring explaining that it decides whether a parsed `name`/`separator` pair from a `str.partition(": ")` call represents a genuine mutant result line, as opposed to a warning or blank line. Inside the helper, assign each sub-condition to a clearly named local variable (e.g. `has_separator`, `is_single_token`, `is_mutant_name`) before combining them in the returned boolean expression. Update `parse_results` to call `if not _is_mutant_result_line(name, separator): continue`. Do not change parsing behaviour or the `MutantResult` construction. Run `workflow_scripts/tests/test_mutation_run_mutmut.py` afterwards to confirm no regressions.

codescene-access[bot]

This comment was marked as outdated.

leynos pushed a commit that referenced this pull request Jul 4, 2026
Two advisory rules failed the quality gate on PR #319:

- `build_arguments` in `mutation_run_cargo.py` took seven keyword
  parameters. Introduce a frozen `MutantsInvocation` dataclass as a
  parameter object and update the CLI entry point and tests.
- `parse_results` in `mutation_run_mutmut.py` used a three-clause
  compound conditional. Extract `_parse_result_line`, which classifies
  one line with simple, named checks and returns `None` for noise.

Behaviour is unchanged; all 47 mutation-script tests pass, and
check-fmt, typecheck, lint, and the full suite remain green.
leynos added 9 commits July 4, 2026 22:04
Plan two reusable workflows — `mutation-cargo.yml` (cargo-mutants) and
`mutation-mutmut.yml` (mutmut) — generalizing the workflow proven in
leynos/wireframe (ADR-007 there): daily cron with a change-detection
guard, file-scoped mutation runs, informative exit-code policy, and
job-summary survivor reporting, never gating pull requests.

Seed the plan with the wireframe findings (`--output` nesting, exit-code
taxonomy, `--in-place`, cron drift, companion-crate false survivors,
env-block injection hardening) and the house conventions: helper logic
as tested Python scripts under `workflow_scripts/`, act + pytest
black-box integration tests with stubbed mutation tools, tool version
pins living beside the report parsers, and caller docs mirroring the
dependabot-automerge page.

Stage A researches mutmut 3.x semantics before the Python workflow is
designed; Stage G validates both workflows against pilot repositories
and records the confirmed tool versions.
Capture measurements from the first full wireframe dispatch run: 1,821
mutants, 84s + 120s baseline, 361s auto timeout, ~28s per mutant —
roughly 14–15 single-job hours, beyond GitHub's 360-minute job ceiling.

Design consequences:

- Add configurable sharding to `mutation-cargo.yml`: `shard-count`
  (default 6) fans full dispatch runs out to a matrix of
  `cargo mutants --shard k/N` legs (~2.5–3 hours each at wireframe
  scale), with a merge-and-summarize job combining shard artefacts.
  Scoped daily runs stay single-shard, since each shard re-pays the
  baseline cost for a handful of mutants.
- Add an `exclude-globs` input: the run's survivors mixed genuine
  assertion gaps with scaffolding noise (example codecs, test helpers),
  so callers need a supported way to exclude such paths.

Also record that `--in-place` cache reuse works as intended and that
`RUSTFLAGS: -D warnings` produced no spurious unviable mutants.
Implement Stage B of the mutation-testing ExecPlan:

- `workflow_scripts/mutation_detect_changes.py`: change-detection guard
  shared by both planned workflows. Buckets files changed within the
  window (commit timestamps, tip-existence filtered) into targets and
  emits a matrix: scoped scheduled runs stay single-shard; full
  dispatch runs fan the root target across `shard-count` shards.
- `workflow_scripts/mutation_run_cargo.py`: builds the cargo-mutants
  invocation (`--in-place`, scoped `--file` args, `--shard k/N`,
  `--exclude` globs, shell-lexed extra args) and applies the
  informational exit-code contract: 0/2/3 succeed, 1/4/70 and unknowns
  fail with the tool's code.
- `workflow_scripts/mutation_summarize_cargo.py`: merges shard
  `outcomes.json` artefacts per target and renders counts plus a
  surviving-mutants table to the job summary. Field names validated
  against cargo-mutants 27.x source; the parser and the workflow's
  version pin (default 27.1.0) travel together.
- `.github/workflows/mutation-cargo.yml`: detect → sharded mutants
  matrix → summarize, reusing the dependabot-automerge OIDC
  source-resolution and `ACT` bypass pattern per job, with SHA-pinned
  third-party actions and least-privilege permissions.

36 unit tests cover bucketing, window edges, deleted-file filtering,
dispatch bypass, matrix shapes, argument construction, exit-code
mapping (via a fake `cargo` on PATH), shard-report merging, and
summary rendering.
Implement Stage C of the mutation-testing ExecPlan:

- `workflow_scripts/mutation_run_mutmut.py`: runs mutmut in the
  caller's uv-managed project (injected via `uv run --with
  mutmut==<pin>`), translating changed files to module-glob mutant
  names because mutmut 3.x rejects file paths as run arguments. Parses
  `mutmut results --all true` into status counts and a survivors table
  appended to the job summary, and writes the raw listing for the
  artefact. A non-zero `mutmut run` (failing baseline) fails the step;
  survivors do not, matching mutmut's own exit-code behaviour.
- `.github/workflows/mutation-mutmut.yml`: single job — mutmut has no
  shard equivalent, so no matrix or cross-job merge — with an inline
  detect step reusing `mutation_detect_changes.py` under a `*.py`
  pathspec, the OIDC source-resolution and `ACT` bypass pattern, and
  artefact upload of the results listing plus `mutmut-stats.json`.

Callers must be uv-managed projects with `[tool.mutmut]` configured;
pip-only projects are out of scope for the first release.

11 unit tests cover module-glob translation (src layout, `__init__.py`
collapse, non-Python inputs), results parsing, summary rendering, and
the CLI paths (scoped run, failing baseline, empty scope) via a fake
`uv` on PATH.
Implement Stage D of the mutation-testing ExecPlan: wrapper workflows
(`test-mutation-cargo.yml`, `test-mutation-mutmut.yml`) call the
reusable workflows locally with never-matching path prefixes, so
scheduled runs deterministically take the change-detection skip path.
The act-driven tests assert the real detection script reports
`has_changes=False` and that no mutation tool is invoked.

The mutation-run path is deliberately not act-tested: the harness
cannot inject stub binaries onto a `workflow_call` job's PATH inside
the container, so that path is covered by the helper-script unit tests
and by pilot-repository validation (Stage G). Recorded in the ExecPlan
Decision Log, revising the earlier stubbed-tools intent.
Implement Stage E of the mutation-testing ExecPlan: caller-facing
documentation for `mutation-cargo.yml` and `mutation-mutmut.yml`
mirroring the dependabot-automerge page — behaviour (guard, sharding,
exit-code contract), required permissions, pinned-SHA usage examples,
inputs tables, triage caveats (false survivors in companion crates,
equivalent mutants, "no tests" semantics), and local validation. Add
both workflows to the README reusable-workflows index.
Record Stage F evidence (all deterministic gates green via scrutineer;
CodeRabbit review with zero findings; act tests environment-blocked
locally) and write the retrospective. Stage G pilot validation remains,
gated on merge so callers can pin a SHA.
The fake `cargo`/`uv` helpers are POSIX shell scripts, so on Windows
runners PATH lookup fell through to the real tools: real `cargo` failed
with exit 101 (`no such command: mutants`), and the mutmut
failing-baseline test passed only by coincidence because real mutmut
exits 1 on Windows ("please use the WSL").

Mark every shim-dependent test with
`skipif(sys.platform == "win32", reason="fake ... helper emits POSIX
sh")`, matching the generate-coverage precedent — including the
coincidentally passing one. The reusable workflows only run on
`ubuntu-latest`, so Windows coverage of the run wrappers carries no
signal. Platform-independent tests (argument construction, exit-code
classification, parsing, rendering, validation errors) still run
everywhere.

Record the finding in the ExecPlan's Surprises & Discoveries.
Two advisory rules failed the quality gate on PR #319:

- `build_arguments` in `mutation_run_cargo.py` took seven keyword
  parameters. Introduce a frozen `MutantsInvocation` dataclass as a
  parameter object and update the CLI entry point and tests.
- `parse_results` in `mutation_run_mutmut.py` used a three-clause
  compound conditional. Extract `_parse_result_line`, which classifies
  one line with simple, named checks and returns `None` for noise.

Behaviour is unchanged; all 47 mutation-script tests pass, and
check-fmt, typecheck, lint, and the full suite remain green.
codescene-access[bot]

This comment was marked as outdated.

CodeScene's advisory gate flagged file-level Overall Code Complexity in
`mutation_run_mutmut.py` after the previous round fixed the individual
findings. Flatten the branchy functions by extraction, without
behaviour change:

- `_module_glob_for` performs the single-file glob translation;
  `files_to_module_globs` becomes a de-duplicating comprehension.
- `_render_survivor_table` isolates the survivors table;
  `render_summary` composes sections.
- `_run_mutation_testing` and `_publish_results` split the CLI entry
  point's run and reporting phases.

All 47 mutation-script tests pass; check-fmt, typecheck, lint, and the
full suite remain green.
@leynos leynos force-pushed the add-mutation-testing-workflows branch from 05951dd to 95a2b6a Compare July 4, 2026 20:09
codescene-access[bot]

This comment was marked as outdated.

@leynos

leynos commented Jul 5, 2026

Copy link
Copy Markdown
Owner Author

@coderabbitai Have the following failed checks now been resolved?

If further work is required, please provide an AI agent prompt for the remaining work to be done to address these failures.

Do not treat warnings as optional or aspirational. Where a change is out of scope for this PR, propose a GitHub issue unless one exists already. (Treat documentation and validation coverage as in scope).

User-Facing Documentation ⚠️ Warning docs/users-guide.md is absent; the new mutation workflows are only signposted in README and workflow-specific docs, not in the required user guide. Add a users guide page covering the new mutation-testing workflows, their purpose, and getting-started links, then keep any translated guides in sync.

@coderabbitai

coderabbitai Bot commented Jul 5, 2026

Copy link
Copy Markdown
Contributor

Tip

For best results, initiate chat on the files or code changes.

🧩 Analysis chain

🏁 Script executed:

#!/bin/bash
fd users-guide.md
echo "---"
fd -g '*users-guide*'
echo "---readme docs section---"
sed -n '1,50p' README.md | grep -n -i "guide" 
echo "---check build_arguments---"
rg -n "def build_arguments" -A 20 workflow_scripts/mutation_run_cargo.py
echo "---check dataclass---"
rg -n "CargoMutantsRunConfig" workflow_scripts/mutation_run_cargo.py workflow_scripts/tests/test_mutation_run_cargo.py
echo "---check parse_results---"
rg -n "_is_mutant_result_line|def parse_results" -A 15 workflow_scripts/mutation_run_mutmut.py
echo "---list docs dir---"
fd . docs -t f

Length of output: 3548


🏁 Script executed:

#!/bin/bash
sed -n '140,200p' workflow_scripts/mutation_run_mutmut.py
echo "---readme table---"
sed -n '20,45p' README.md
echo "---check MutantsInvocation fields---"
rg -n "class MutantsInvocation" -A 15 workflow_scripts/mutation_run_cargo.py
echo "---search existing github issues for users-guide---"
gh issue list --repo leynos/shared-actions --search "users-guide" --state all
gh issue list --repo leynos/shared-actions --search "user guide" --state all

Length of output: 4962


@leynos Here is the status of each item.

Code-health findings: both resolved.

  • workflow_scripts/mutation_run_cargo.py: build_arguments (Line 113) now takes a single MutantsInvocation dataclass (Line 83–98ish) instead of seven arguments. Functionally identical to the CargoMutantsRunConfig suggestion, just named differently — no further action needed.
  • workflow_scripts/mutation_run_mutmut.py: parse_results (Line 154) now delegates to a new _parse_result_line helper that encapsulates the compound guard (not separator, is_mutant_name). The complex conditional is decomposed — no further action needed.

Documentation warning: not resolved.

docs/users-guide.md still does not exist in the repository (only docs/cmd-mox-users-guide.md and docs/syspath-hack-users-guide.md are present, and these cover unrelated actions). The mutation-testing workflows are signposted only in the README table and their own workflow-specific guides (docs/mutation-cargo-workflow.md, docs/mutation-mutmut-workflow.md), not in a top-level user guide. I found no existing issue tracking this. Treat this as in scope for the current PR, and address it now rather than defer it.

Here is a prompt for an AI coding agent to close this gap.

Add a top-level `docs/users-guide.md` for the leynos/shared-actions repository, or extend an existing equivalent top-level guide if one is intended to serve this purpose (search the repository first to confirm no such file exists under a different name before creating a new one).

The new guide must:
1. Introduce the two new scheduled mutation-testing reusable workflows added in this PR: `.github/workflows/mutation-cargo.yml` (cargo-mutants, for Rust repositories) and `.github/workflows/mutation-mutmut.yml` (mutmut, for uv-managed Python repositories).
2. State their purpose plainly: informational, non-blocking, scheduled mutation testing that reports surviving mutants via job summaries and artefacts, with a daily change-detection guard that skips quiet days.
3. Provide getting-started links: point readers to `docs/mutation-cargo-workflow.md` and `docs/mutation-mutmut-workflow.md` for full usage examples, inputs, and required permissions (`contents: read`, `id-token: write`).
4. Follow the structure and tone of any other reusable-workflow entries already summarised in the README's "Reusable workflows" table, so the guide reads consistently with existing documentation (check `docs/dependabot-automerge-workflow.md` for the sibling pattern already listed in that table).
5. Add a corresponding entry/link to `docs/users-guide.md` from the README where other user-facing guides are indexed, if such an index exists; otherwise ensure `README.md`'s existing links remain correct and add a reference to the new guide.
6. Check for any translated copies of a user guide in the repository (e.g. under a locale-suffixed filename or a translations directory) and update them to reference the same new content, keeping them in sync. If no translations exist, state that in the PR description so this is not missed later.
7. Run `make markdownlint` (or the project's equivalent Markdown lint target) and any documentation link-check target to confirm the new file passes validation.

If you would prefer to defer this to a follow-up PR instead, let me know and I will open a GitHub issue with this same scope.


✏️ Learnings added
Learnt from: leynos
URL: https://github.com/leynos/shared-actions/pull/319

Timestamp: 2026-07-05T20:39:18.814Z
Learning: In leynos/shared-actions, `workflow_scripts/mutation_run_cargo.py`'s parameter-object refactor for `build_arguments` uses a dataclass named `MutantsInvocation` (not `CargoMutantsRunConfig`); `workflow_scripts/mutation_run_mutmut.py`'s `parse_results` decomposes its conditional via a `_parse_result_line` helper (not `_is_mutant_result_line`). Reference these actual symbol names in future reviews of these files.

Note: Learnings are effective only in the context of similar code segments. To apply general coding guidelines, please configure review instructions. You can manage existing learnings in the app.

@leynos

leynos commented Jul 5, 2026

Copy link
Copy Markdown
Owner Author

@coderabbitai please create a new GitHub issue for the following:

Documentation warning: not resolved.

docs/users-guide.md still does not exist in the repository (only docs/cmd-mox-users-guide.md and docs/syspath-hack-users-guide.md are present, and these cover unrelated actions). The mutation-testing workflows are signposted only in the README table and their own workflow-specific guides (docs/mutation-cargo-workflow.md, docs/mutation-mutmut-workflow.md), not in a top-level user guide. I found no existing issue tracking this. Treat this as in scope for the current PR, and address it now rather than defer it.

Here is a prompt for an AI coding agent to close this gap.

Add a top-level `docs/users-guide.md` for the leynos/shared-actions repository, or extend an existing equivalent top-level guide if one is intended to serve this purpose (search the repository first to confirm no such file exists under a different name before creating a new one).

The new guide must:
1. Introduce the two new scheduled mutation-testing reusable workflows added in this PR: `.github/workflows/mutation-cargo.yml` (cargo-mutants, for Rust repositories) and `.github/workflows/mutation-mutmut.yml` (mutmut, for uv-managed Python repositories).
2. State their purpose plainly: informational, non-blocking, scheduled mutation testing that reports surviving mutants via job summaries and artefacts, with a daily change-detection guard that skips quiet days.
3. Provide getting-started links: point readers to `docs/mutation-cargo-workflow.md` and `docs/mutation-mutmut-workflow.md` for full usage examples, inputs, and required permissions (`contents: read`, `id-token: write`).
4. Follow the structure and tone of any other reusable-workflow entries already summarised in the README's "Reusable workflows" table, so the guide reads consistently with existing documentation (check `docs/dependabot-automerge-workflow.md` for the sibling pattern already listed in that table).
5. Add a corresponding entry/link to `docs/users-guide.md` from the README where other user-facing guides are indexed, if such an index exists; otherwise ensure `README.md`'s existing links remain correct and add a reference to the new guide.
6. Check for any translated copies of a user guide in the repository (e.g. under a locale-suffixed filename or a translations directory) and update them to reference the same new content, keeping them in sync. If no translations exist, state that in the PR description so this is not missed later.
7. Run `make markdownlint` (or the project's equivalent Markdown lint target) and any documentation link-check target to confirm the new file passes validation.

@coderabbitai

coderabbitai Bot commented Jul 5, 2026

Copy link
Copy Markdown
Contributor

Tip

For best results, initiate chat on the files or code changes.

🧩 Analysis chain

🏁 Script executed:

#!/bin/bash
set -euo pipefail

gh issue create \
  --repo leynos/shared-actions \
  --assignee leynos \
  --title "Add top-level docs/users-guide.md covering mutation-testing workflows" \
  --body "$(cat <<'EOF'
## Summary

`docs/users-guide.md` does not exist in the repository. Only `docs/cmd-mox-users-guide.md` and `docs/syspath-hack-users-guide.md` are present, and these cover unrelated actions. The two new scheduled mutation-testing reusable workflows added in PR `#319` (`.github/workflows/mutation-cargo.yml` for cargo-mutants, and `.github/workflows/mutation-mutmut.yml` for mutmut) are signposted only in the README's "Reusable workflows" table and their own workflow-specific guides (`docs/mutation-cargo-workflow.md`, `docs/mutation-mutmut-workflow.md`), not in a top-level user guide.

## Rationale

Callers adopting reusable workflows benefit from a single top-level entry point that introduces available workflows and links onward to detailed usage docs, rather than having to discover them solely from the README table.

## Required changes

- Add a top-level `docs/users-guide.md` (or confirm/extend an existing equivalent top-level guide, if one is intended to serve this purpose — search first before creating a new file).
- Introduce the two mutation-testing workflows: `.github/workflows/mutation-cargo.yml` (cargo-mutants, Rust) and `.github/workflows/mutation-mutmut.yml` (mutmut, uv-managed Python).
- State their purpose plainly: informational, non-blocking, scheduled mutation testing that reports surviving mutants via job summaries and artefacts, with a daily change-detection guard that skips quiet days.
- Link onward to `docs/mutation-cargo-workflow.md` and `docs/mutation-mutmut-workflow.md` for full usage examples, inputs, and required permissions (`contents: read`, `id-token: write`).
- Match the structure and tone of existing reusable-workflow documentation (see `docs/dependabot-automerge-workflow.md` for the sibling pattern already listed in the README table).
- Cross-link the new guide from `README.md` where user-facing guides are indexed.
- Check for translated copies of a user guide (locale-suffixed filenames or a translations directory) and keep them in sync; if none exist, note that explicitly.
- Run markdownlint and any documentation link-check target to confirm the new file passes validation.

## Affected areas

- `docs/users-guide.md` (new)
- `README.md`
- `docs/mutation-cargo-workflow.md`, `docs/mutation-mutmut-workflow.md` (cross-links only)

## Acceptance criteria

- [ ] `docs/users-guide.md` exists, introduces both mutation-testing workflows, and links to their dedicated guides.
- [ ] README links to the new guide correctly.
- [ ] Any translated guides are updated or their absence is confirmed.
- [ ] Markdownlint and documentation validation pass.

## Backlinks

- Pull request: https://github.com/leynos/shared-actions/pull/319
- Originating comment: https://github.com/leynos/shared-actions/pull/319#issuecomment (see review thread requesting this issue)
- Requested by: `@leynos`
EOF
)"

Length of output: 5020


🏁 Script executed:

#!/bin/bash
set -euo pipefail

gh issue create \
  --repo leynos/shared-actions \
  --assignee leynos \
  --title "Add top-level docs/users-guide.md covering mutation-testing workflows" \
  --body "$(cat <<'EOF'
## Summary

`docs/users-guide.md` does not exist in the repository. Only `docs/cmd-mox-users-guide.md` and `docs/syspath-hack-users-guide.md` are present, and these cover unrelated actions. The two new scheduled mutation-testing reusable workflows added in PR `#319` (`.github/workflows/mutation-cargo.yml` for cargo-mutants, and `.github/workflows/mutation-mutmut.yml` for mutmut) are signposted only in the README's "Reusable workflows" table and their own workflow-specific guides (`docs/mutation-cargo-workflow.md`, `docs/mutation-mutmut-workflow.md`), not in a top-level user guide.

## Rationale

Callers adopting reusable workflows benefit from a single top-level entry point that introduces available workflows and links onward to detailed usage docs, rather than having to discover them solely from the README table.

## Required changes

- Add a top-level `docs/users-guide.md` (or confirm/extend an existing equivalent top-level guide, if one is intended to serve this purpose — search first before creating a new file).
- Introduce the two mutation-testing workflows: `.github/workflows/mutation-cargo.yml` (cargo-mutants, Rust) and `.github/workflows/mutation-mutmut.yml` (mutmut, uv-managed Python).
- State their purpose plainly: informational, non-blocking, scheduled mutation testing that reports surviving mutants via job summaries and artefacts, with a daily change-detection guard that skips quiet days.
- Link onward to `docs/mutation-cargo-workflow.md` and `docs/mutation-mutmut-workflow.md` for full usage examples, inputs, and required permissions (`contents: read`, `id-token: write`).
- Match the structure and tone of existing reusable-workflow documentation (see `docs/dependabot-automerge-workflow.md` for the sibling pattern already listed in the README table).
- Cross-link the new guide from `README.md` where user-facing guides are indexed.
- Check for translated copies of a user guide (locale-suffixed filenames or a translations directory) and keep them in sync; if none exist, note that explicitly.
- Run markdownlint and any documentation link-check target to confirm the new file passes validation.

## Affected areas

- `docs/users-guide.md` (new)
- `README.md`
- `docs/mutation-cargo-workflow.md`, `docs/mutation-mutmut-workflow.md` (cross-links only)

## Acceptance criteria

- [ ] `docs/users-guide.md` exists, introduces both mutation-testing workflows, and links to their dedicated guides.
- [ ] README links to the new guide correctly.
- [ ] Any translated guides are updated or their absence is confirmed.
- [ ] Markdownlint and documentation validation pass.

## Backlinks

- Pull request: https://github.com/leynos/shared-actions/pull/319
- Requested by: `@leynos`
EOF
)"

Length of output: 3119


@leynos Issue created: #321 — "Add top-level docs/users-guide.md covering mutation-testing workflows", assigned to you, with the full scope, rationale, affected areas, and acceptance criteria carried over from your comment, plus a backlink to this PR.

(=^・ω・^=)

@leynos leynos merged commit 47aea18 into main Jul 5, 2026
27 checks passed
@leynos leynos deleted the add-mutation-testing-workflows branch July 5, 2026 20:41
This was referenced Jul 6, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants