Add scheduled mutation testing to generated repos#52
Conversation
Add a mutation-testing caller workflow to the template so newly generated Rust projects run scheduled mutation testing by default. The workflow is a thin caller of the shared mutation-cargo reusable workflow in leynos/shared-actions, pinned to a full commit SHA in line with the template's action-pinning convention. The workflow runs daily at 09:15 UTC with a comment advising maintainers to stagger the slot per repository, and supports manual dispatch. It defaults the token to no scopes, with the job opting in to contents: read and id-token: write. The extra-args input mirrors the CI test baseline (--all-features); workspace inputs are omitted as the generated project is a single crate. The file is plain YAML rather than Jinja-templated, matching ci.yml and act-validation.yml, as it needs no template variables. The generated repository-layout document is updated to list the new workflow.
![sourcery-ai[bot]](https://avatars.githubusercontent.com/in/48477?s=60&v=4){kind=small}
|
Warning Review limit reachedYou’ve reached a temporary PR review limit under our Fair Usage Limits Policy. Next review available in: 8 minutes Your organization has reached its usage spending cap. Adjust your spending cap in the billing tab. How can I continue?After more reviews become available, a review can be triggered using the To avoid repeated limits, reduce automatic review volume by pausing incremental auto-reviews earlier, using label-based review opt-in, excluding WIP or generated PR titles, or requesting reviews manually when the PR is ready. If your team needs uninterrupted high-volume reviews, an organization admin can enable usage-based reviews. How do review limits work?CodeRabbit enforces per-developer PR review limits for each organization. Most developers receive the normal plan review availability. For paid Pro and Pro+ PR reviews, CodeRabbit uses adaptive limits for sustained high-volume activity. When a developer's recent PR review activity reaches the 95th percentile or higher among CodeRabbit users, additional reviews become available more gradually as earlier reviews age out of the rolling window. Please refer docs for additional details. Review details⚙️ Run configurationConfiguration used: Organization UI Review profile: ASSERTIVE Plan: Pro Plus Run ID: 📒 Files selected for processing (2)
Note Reviews pausedIt looks like this branch is under active development. To avoid overwhelming you with review comments due to an influx of new commits, CodeRabbit has automatically paused this review. You can configure this behavior by changing the Use the following commands to manage reviews:
Use the checkboxes below for quick actions:
WalkthroughAdds a new GitHub Actions workflow, ChangesMutation testing workflow
Possibly related PRs
Poem
Caution Pre-merge checks failedPlease resolve all errors before merging. Addressing warnings is optional.
❌ Failed checks (1 error, 3 warnings)
✅ Passed checks (16 passed)
✨ Finishing Touches🧪 Generate unit tests (beta)
Comment |
![chatgpt-codex-connector[bot]](https://avatars.githubusercontent.com/in/1144995?s=60&v=4){kind=small}
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 5cb3aa8dbe
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
The generated .cargo/config.toml links Linux dev targets with clang and -fuse-ld=mold, and the generated ci.yml installs mold before any cargo command, but the mutation workflow's mutants jobs had no way to install it, so the cargo-mutants baseline would fail to link on a bare ubuntu-latest runner. Bump the shared workflow pin to 0c8bd605f8966a004decb0b54f717a745f51bc57, which introduces the setup-commands input, and use it to install clang, lld, and mold exactly as the generated ci.yml does. The install is unconditional, matching ci.yml's own convention: CI always runs on Linux runners, and ci.yml installs the packages regardless of the dev_target chosen at generation time, so the workflow stays a plain copied file rather than becoming Jinja-templated.
|
Review feedback addressed in facc38a: the generated
Re-validated: |
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
|
@coderabbitai review |
✅ Action performedReview finished.
|
There was a problem hiding this comment.
Actionable comments posted: 1
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In `@template/.github/workflows/mutation-testing.yml`:
- Around line 1-47: The workflow contract for mutation testing is not covered by
assertions, so changes to the pinned reusable-workflow SHA, the setup package
list, or the job permissions could slip through. Extend the existing
tooling-contract tests around mutation-testing.yml to verify the job uses the
expected pinned ref in the mutation job, the setup-commands installs the
required packages, and the mutation job permissions remain scoped to contents:
read and id-token: write.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Organization UI
Review profile: ASSERTIVE
Plan: Pro Plus
Run ID: 27643c2a-2103-4b4a-9c4f-1d5db8d3a601
📒 Files selected for processing (2)
template/.github/workflows/mutation-testing.ymltemplate/docs/repository-layout.md.jinja
| name: Mutation testing | ||
|
|
||
| # Thin caller of the shared mutation-testing reusable workflow; caller | ||
| # guide: leynos/shared-actions docs/mutation-cargo-workflow.md. | ||
| # Scheduled runs mutate only files changed within the detection window; | ||
| # manual dispatch runs mutate everything, fanned out across shards. To | ||
| # test a branch, select it in the Actions "Run workflow" control. | ||
|
|
||
| 'on': | ||
| schedule: | ||
| # Daily, 09:15 UTC. Stagger this slot when adopting the workflow in a | ||
| # new repository: the estate already occupies 03:05-09:05 UTC, so pick | ||
| # an unclaimed slot to avoid concurrent runs across repositories. | ||
| - cron: "15 9 * * *" | ||
| workflow_dispatch: | ||
|
|
||
| # Default the token to no scopes; the job opts in explicitly. | ||
| permissions: {} | ||
|
|
||
| # Serialize runs per ref: a dispatch during a scheduled run (or vice | ||
| # versa) queues rather than racing. Runs are informational, so queued | ||
| # runs wait instead of cancelling. | ||
| concurrency: | ||
| group: mutation-testing-${{ github.ref }} | ||
| cancel-in-progress: false | ||
|
|
||
| jobs: | ||
| mutation: | ||
| permissions: | ||
| contents: read | ||
| id-token: write # OIDC workflow-source resolution | ||
| uses: leynos/shared-actions/.github/workflows/mutation-cargo.yml@0c8bd605f8966a004decb0b54f717a745f51bc57 | ||
| with: | ||
| # Match the CI baseline (`make test` runs --all-targets | ||
| # --all-features) so the test command exercised against mutants | ||
| # mirrors what gates pull requests. | ||
| extra-args: "--all-features" | ||
| # .cargo/config.toml links Linux dev targets with clang and | ||
| # -fuse-ld=mold, so the mutants jobs need the same packages that | ||
| # ci.yml installs before any cargo command runs on a bare | ||
| # ubuntu-latest runner. The reusable workflow executes these under | ||
| # bash with `set -euo pipefail`. | ||
| setup-commands: | | ||
| export DEBIAN_FRONTEND=noninteractive | ||
| # mold accelerates dev builds; coverage uses lld for llvm-tools compatibility. | ||
| sudo apt-get update \ | ||
| && sudo apt-get install --yes --no-install-recommends clang lld mold |
There was a problem hiding this comment.
📐 Maintainability & Code Quality | 🟡 Minor | ⚡ Quick win
🧩 Analysis chain
🏁 Script executed:
#!/bin/bash
# Description: Check whether a contract test references mutation-testing.yml.
rg -n 'mutation-testing' tests/ --type=pyRepository: leynos/agent-template-rust
Length of output: 164
🏁 Script executed:
#!/bin/bash
set -euo pipefail
printf '\n## files under tests\n'
git ls-files tests | sed -n '1,200p'
printf '\n## workflow contract helpers\n'
rg -n 'assert_.*workflow|ci.yml|mutation-testing|tooling_contracts' tests/ --type=py -A 3 -B 3
printf '\n## helper file outline\n'
ast-grep outline tests/helpers/tooling_contracts/workflows.py --view expandedRepository: leynos/agent-template-rust
Length of output: 11403
🏁 Script executed:
#!/bin/bash
set -euo pipefail
printf '\n## workflow helper file\n'
if [ -f tests/helpers/tooling_contracts/workflows.py ]; then
cat -n tests/helpers/tooling_contracts/workflows.py | sed -n '1,260p'
else
echo 'missing: tests/helpers/tooling_contracts/workflows.py'
fi
printf '\n## tests importing workflow helpers\n'
rg -n 'tooling_contracts\.workflows|assert_ci_coverage_action_contract|assert_.*contract' tests/ --type=py -A 2 -B 2
printf '\n## any references to workflow filenames in tests\n'
rg -n 'ci\.yml|mutation-testing\.yml|workflows/' tests/ --type=py -A 2 -B 2Repository: leynos/agent-template-rust
Length of output: 26294
Add a contract assertion for template/.github/workflows/mutation-testing.yml. Extend the existing tooling-contract tests so the pinned SHA, setup-commands package list, and scoped job permissions stay locked to the workflow contract.
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In `@template/.github/workflows/mutation-testing.yml` around lines 1 - 47, The
workflow contract for mutation testing is not covered by assertions, so changes
to the pinned reusable-workflow SHA, the setup package list, or the job
permissions could slip through. Extend the existing tooling-contract tests
around mutation-testing.yml to verify the job uses the expected pinned ref in
the mutation job, the setup-commands installs the required packages, and the
mutation job permissions remain scoped to contents: read and id-token: write.
Source: Coding guidelines
|
@coderabbitai review |
✅ Action performedReview finished.
|
2 participants
Summary
Adds
template/.github/workflows/mutation-testing.ymlso newly generated Rust repositories get scheduled mutation testing by default, matching the pattern already rolled out across the estate (for exampleleynos/mapsplice).The workflow is a thin caller of the shared reusable workflow
leynos/shared-actions/.github/workflows/mutation-cargo.yml, pinned to2b09d10192627fd6e1034e7c12625dd266b45503. Caller guide: https://github.com/leynos/shared-actions/blob/main/docs/mutation-cargo-workflow.md.Details:
workflow_dispatch. The default cron is15 9 * * *; the estate already occupies the 03:05–09:05 UTC slots, and an inline comment tells maintainers of generated repositories to stagger their slot to avoid concurrent runs across the estate.permissions: {}; themutationjob opts in tocontents: readandid-token: write(OIDC workflow-source resolution).mutation-testing-${{ github.ref }}withcancel-in-progress: false, so dispatches queue behind scheduled runs rather than racing or cancelling.extra-args: "--all-features"mirrors the template CI test baseline (make testruns with--all-targets --all-features). Workspace-related inputs are omitted because the generated project is a single root crate.ci.ymlandact-validation.yml, as it needs no template variables; this also sidesteps the${{ }}Jinja/Actions syntax collision.docs/repository-layout.mdis updated to list the new workflow.Validation
copier copy --defaultsrenders cleanly and the workflow is copied verbatim.actionlintpasses on the rendered file.make test: 47 passed, 1 skipped; the 3 failures intests/test_template/test_lint_targets.pyare pre-existing onmainand unrelated to this change.