Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
32 changes: 30 additions & 2 deletions .github/workflows/drift-guard.yml
Original file line number Diff line number Diff line change
Expand Up @@ -234,7 +234,7 @@ jobs:
- name: Self-test the duplicate-ctest-name checker
run: bash scripts/test_check_ctest_name_collisions.sh

# ── The scheduled mutation gate's own self-test (morph#408) ──────────────
# ── The mutation gates: a self-test (morph#408) and a citation gate (morph#608) ──
#
# scripts/check_mutation_regression.py is the gate .github/workflows/mutation.yml
# runs after scripts/mutation.sh, on a schedule rather than per-PR (that
Expand All @@ -253,14 +253,42 @@ jobs:
# successful run -- would be indistinguishable from a healthy one without
# this self-test.
mutation-regression-lint:
name: Mutation-regression checker self-test
name: Mutation gates (regression self-test, survivor citations)
runs-on: ubuntu-24.04
steps:
- uses: actions/checkout@v4

- name: Self-test the mutation-regression checker
run: python3 scripts/check_mutation_regression.py --self-test

# scripts/check_mutation_survivors.py, and unlike everything else in this
# file this one is the *gate itself*, not only its self-test. It reads no
# coverage data and no Mull report -- its inputs are
# scripts/mutation_survivors.json and the headers in the tree -- so there
# is nothing to defer it to a build leg or to the scheduled campaign for.
#
# It belongs per-PR precisely because what it audits rots per-PR. The
# triage record cites each equivalent mutant by `{file, line, source}`,
# and an ordinary edit anywhere above one of those lines moves it. By the
# time morph#608 measured, 5 of the 7 entries pointed at the wrong line
# and include/morph/core/backend.hpp's was off by 479 -- landing in the
# middle of an unrelated function, for a reader the mutation gate's own
# failure message had sent there. Nothing failed, because nothing looked:
# check_mutation_regression.py compares survivor *counts* against
# scripts/mutation_baseline.json and never opens the triage file at all.
#
# The resolution is check_branch_coverage.py's
# resolve_allowlist_source_line(), shared rather than reimplemented --
# scripts/branch_partial_allowlist.json and
# scripts/error_path_allowlist.json are hardened by that same function,
# against the same defect found three times over (morph#349, morph#355,
# morph#419).
- name: Audit the mutation-survivor citations
run: python3 scripts/check_mutation_survivors.py

- name: Self-test the mutation-survivor citation gate
run: python3 scripts/check_mutation_survivors.py --self-test

# ── The coverage-object gate's own self-test ───────────────────────────
# The gate itself (scripts/check_coverage_objects.sh) runs inside
# scripts/coverage.sh, in ci.yml's linux-coverage job, because it compares the
Expand Down
3 changes: 2 additions & 1 deletion docs/spec/testing_charter.md
Original file line number Diff line number Diff line change
Expand Up @@ -74,6 +74,7 @@ floor means anything.
| Compile-time contract checks | `tests/compile_checks/` | Every configure that reaches `tests/CMakeLists.txt` | `FATAL_ERROR` at configure time |
| Multiple independent harnesses | Catch2 suite + `scripts/scenario/` (wire-level scenario corpus against real rung servers) | `tests/`, local scenario runs | Catch2 assertions; `scripts/scenario/scenario_coverage.py`'s drift gate (self-tested in `drift-guard.yml`) |
| Error-path instrumentation | `scripts/check_error_path_coverage.py` (morph#406) | Local / on demand; self-test only in `drift-guard.yml` today | Its own self-test; not yet wired as a build-blocking gate (see "What is unenforced") |
| Mutation-triage citations still point at the code they triage | `scripts/check_mutation_survivors.py` (morph#608) | `drift-guard` (the gate itself, per-PR — it needs no build and no Mull report) | The gate: every `{file, line, source}` entry in `scripts/mutation_survivors.json` must resolve to the line it names, via `check_branch_coverage.py`'s `resolve_allowlist_source_line()`. Audits the structured entries only; the same file's free-text citations are counted and reported, not gated (morph#613) |

## Declined techniques, and why

Expand Down Expand Up @@ -196,6 +197,6 @@ Named honestly rather than folded into the table above as if a check existed:
| [testing_strategy.md](testing_strategy.md) | The opt-in test categories this charter's table cites in detail — fuzz harness, soak tests, load benchmark, adversarial cross-socket run. |
| [error_handling.md](error_handling.md) | The propagation design morph#406's error-path instrument measures test coverage of. |
| `codecov.yml` | The per-subsystem coverage targets and the artifact-audit allowlists (`branch_partial_allowlist.json`, `error_path_allowlist.json`) this charter's guarantee is enforced through. |
| `scripts/mutation_survivors.json` | The triaged survivor list behind this charter's 64.06% mutation-score figure. |
| `scripts/mutation_survivors.json` | The triaged survivor list behind this charter's 64.06% mutation-score figure. Its structured `{file, line, source}` entries are audited per-PR by `scripts/check_mutation_survivors.py` (morph#608); its free-text citations are not (morph#613). |
| `tests/oom_injector.hpp` | The OOM-injection limitation this charter states under "Instrument reach". |
| `examples/common/testkit/fault_proxy.hpp` | The ladder-only fault-injection seam this charter states has no `include/morph`-side equivalent. |
Loading
Loading