Add scheduled mutation testing via the shared workflow#67
Merged
Conversation
Adopt the leynos/shared-actions mutation-cargo reusable workflow as a thin caller, pinned to 859416a90eb3987b46a57682c5d6b8964ad3f0a6, scheduled daily at 09:20 UTC with manual dispatch. The run is informational: it never gates merges. The caller configures only what this repository needs: - extra-args "--all-features", mirroring the CI test baseline (make test runs cargo test --all-targets --all-features); - exclude-globs for test scaffolding (the re-exported src/tests module and tests/support helpers) and the manually invoked golden-snapshot regenerator src/bin/bless_traces.rs, none of which the test suite guards. The default source paths (src/, examples/, benches/) already cover this single-crate layout, so the paths input is omitted, as is --test-workspace (not a workspace). A contract test suite (tests/workflow_contracts) parses the caller with PyYAML and pins the SHA, least-privilege permissions, per-ref non-cancelling concurrency, the cron slot, and the exact with-block, so drift fails CI on the pull request. CI wires it in via a new test-workflow-contracts make target after the Lint step, and .gitignore now covers __pycache__/. Note: the mutation baseline is currently blocked by three doctests in src/heuristics/text.rs that fail under plain cargo test (issue #66); CI misses them because --all-targets skips doctests.
![sourcery-ai[bot]](https://avatars.githubusercontent.com/in/48477?s=60&v=4){kind=small}
Contributor
|
Note Reviews pausedIt looks like this branch is under active development. To avoid overwhelming you with review comments due to an influx of new commits, CodeRabbit has automatically paused this review. You can configure this behavior by changing the Use the following commands to manage reviews:
Use the checkboxes below for quick actions:
WalkthroughThis PR adds a scheduled mutation-testing GitHub Actions workflow that calls a shared reusable workflow with pinned SHA, permissions, and concurrency settings. A new Makefile target and CI step run a PyYAML-based contract test suite verifying the workflow's structure, plus a minor ChangesMutation testing workflow and contract verification
Possibly related PRs
Suggested labels: Poem
🚥 Pre-merge checks | ✅ 5 | ❌ 15❌ Failed checks (15 inconclusive)
✅ Passed checks (5 passed)
✨ Finishing Touches🧪 Generate unit tests (beta)
Comment |
![codescene-access[bot]](https://avatars.githubusercontent.com/in/3755875?s=60&v=4){kind=small}
This comment was marked as outdated.
This comment was marked as outdated.
Sorry, something went wrong.
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
Contributor
There was a problem hiding this comment.
Actionable comments posted: 2
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In `@Makefile`:
- Around line 21-23: The test-workflow-contracts target currently pulls pytest
and pyyaml with ad-hoc version ranges via uv run, which bypasses the project’s
locked dev set. Move pytest and pyyaml into the dev dependency group in
pyproject.toml, then update the Makefile target to run pytest without --with
flags so it uses the reproducible resolved environment.
In `@tests/workflow_contracts/mutation_testing_test.py`:
- Around line 42-44: The repeated calls to _load() in the mutation testing tests
reparses the same workflow YAML in every test, so move that parsing into a
shared module-scoped pytest fixture and inject it into each test instead. Create
the fixture in conftest.py or the shared fixtures area per project convention,
have it return the parsed workflow data once, and update each test_* function to
accept the fixture parameter rather than calling _load() directly. Keep _load()
only if needed as the fixture helper, and ensure all existing tests use the
shared workflow fixture consistently.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Organization UI
Review profile: ASSERTIVE
Plan: Pro Plus
Run ID: 9a6814df-3376-4480-a3be-d169fb220835
📒 Files selected for processing (5)
.github/workflows/ci.yml.github/workflows/mutation-testing.yml.gitignoreMakefiletests/workflow_contracts/mutation_testing_test.py
🔗 Linked repositories identified
CodeRabbit considers these linked repositories for cross-repo context during reviews:
leynos/shared-actions(auto-detected)
Replace the per-test _load() helper with a module-scoped pytest fixture so the caller workflow is parsed once rather than in every test, as CodeRabbit suggested and the repository's test guidance prefers.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Enrols lag-complexity in the estate-wide scheduled mutation testing rollout. This pull request adds a thin caller of the leynos/shared-actions mutation-cargo reusable workflow, pinned to commit
859416a90eb3987b46a57682c5d6b8964ad3f0a6, following the estate template established in leynos/wireframe#572. The run is informational only — a daily scoped guard at 09:20 UTC plus sharded full runs on manual dispatch — and never gates merges.Configuration for this repository
.github/workflows/mutation-testing.yml— the caller. It sets only what this repository needs:extra-args: "--all-features"mirrors the CI test baseline (make testrunscargo test --all-targets --all-features), so feature-gated code (cli,json5,yaml,toml,provider-api) is built and tested against mutants.exclude-globs: "src/tests/**,tests/support/**,src/bin/bless_traces.rs"excludes test scaffolding —src/testsre-exportstests/supportinto the library (via#[path]), so both would otherwise be mutation targets — and the golden-snapshot regeneratorbless_traces, which is invoked manually (make bless) and not guarded by the test suite. Mutants there would only be noise survivors.pathsinput is omitted: the default (src/,examples/,benches/) already covers this single-crate layout.--test-workspaceis omitted because the repository is not a cargo workspace, and nosetup-commandsare needed (no custom linker).tests/workflow_contracts/mutation_testing_test.py— contract tests that parse the caller with PyYAML and pin the SHA, least-privilege permissions, per-ref non-cancelling concurrency, the 09:20 UTC cron slot, and the exactwith:block, so drift fails CI on the pull request rather than surfacing in a scheduled run.Makefile— atest-workflow-contractstarget running the suite viauv runwith pytest and PyYAML..github/workflows/ci.yml— a pinnedastral-sh/setup-uvstep and the contract-test step after Lint in thebuild-testjob..gitignore— now covers__pycache__/.Known limitation: baseline currently blocked (#66)
Baseline verification found that plain
cargo test --all-features— what cargo-mutants runs to validate the unmutated tree — fails on three doctests insrc/heuristics/text.rs, which import thepub(crate)moduleheuristics::textfrom outside the crate. CI stays green because--all-targetsskips doctests. Until #66 is resolved, the scheduled run will report a failed baseline rather than mutation results. The workflow is informational, so enrolment proceeds regardless.Validation
python3 -c "import yaml; …"— both workflow files parse.actionlint— clean onmutation-testing.ymlandci.yml.make test-workflow-contracts— 6 passed.uses:ref to@v1failstest_uses_reference_is_pinned_to_the_documented_sha; restoring the SHA returns the suite to green.