Skip to content

ci: mutation_survivors.json's line hints are unaudited and 5 of 7 are already stale #608

Description

@Yaraslaut

Noticed while working #528, unrelated to that change. Filed rather than folded in.

Summary

scripts/mutation_survivors.json records each triaged equivalent mutant as {file, line, mutator, source, reason}. Nothing audits the line field, and 5 of its 7 line-carrying entries already point at the wrong line on master.

This is the exact rot that scripts/branch_partial_allowlist.json and scripts/error_path_allowlist.json were hardened against — check_branch_coverage.py's resolve_allowlist_source_line() resolves an entry by its source text and fails the gate when the line hint has drifted, citing morph#349 and morph#419 for why line numbers alone rot. mutation_survivors.json carries the same shape of citation with none of that auditing.

Verification status

Reproduced by measurement, on master @ 26bfdb8f (i.e. before the #528 branch, so none of this drift is that change's doing). Method: for each entry, search the file at that revision for a line whose stripped text equals the entry's source, and compare with the entry's line.

Real output:

entries with file/line/source: 7
include/morph/core/bridge.hpp:119 -- text now at line(s) [121] | source='return modelHash ^ (key.sharing.hash_code() + 0x9e3779b9U + (modelHash'
include/morph/core/backend.hpp:749 -- text now at line(s) [1226] | source='aware.reserve(_changeAware.size());'
include/morph/core/wire.hpp:302 -- text now at line(s) [307] | source='out.reserve(text.size());'
include/morph/forms/views.hpp:264 -- text now at line(s) [271] | source='ordered.reserve(propsObj.size());'
include/morph/forms/views.hpp:430 -- text now at line(s) [437] | source='ids.reserve(_providers.size());'
stale: 5 of 7

Every source text still exists and is still unique, so no disposition is wrong — only the coordinates are. backend.hpp:749 is the worst of them: it is off by 477 lines and now lands in the middle of an unrelated function.

What I did not verify: whether the two non-stale entries are non-stale by luck or because they were refreshed. I also did not check the plain-string citations in the same file (e.g. "include/morph/core/backend.hpp:613 -- emitMetric(Metric::registerCount, 1.0)" and ":807 -- emitMetric(Metric::executeInFlight, ...)" in the runs narrative), which are free text rather than structured entries; both look stale by inspection but I did not measure them.

Why it matters, and why it is minor

check_mutation_regression.py compares survivor counts against scripts/mutation_baseline.json; it never reads mutation_survivors.json, so no gate is currently wrong and no CI job fails because of this. The cost is entirely on the human the failure message sends here:

A new survivor means the suite stopped noticing ... triage into scripts/mutation_survivors.json

— who then follows include/morph/core/backend.hpp:749 to code that has nothing to do with the recorded reason. That is the same "a bare suppression is not a disposition" problem morph#404 named, one step removed: the disposition is sound but no longer findable.

Suggested fix

Reuse what already exists rather than inventing a second mechanism: check_branch_coverage.py:199's resolve_allowlist_source_line() is already written to be shared (its own docstring says so, citing morph#406 for the second caller). A small checker that walks mutation_survivors.json's structured entries through it, run in the same self-test job as the other allowlist gates, would fail on drift and print the corrected line — turning this into a mechanical fix-up instead of a silent rot. Refreshing the five current hints is the other half.

What would change the verdict

  • Close it if the line fields in this file are deliberately historical — a record of where the survivor was when it was triaged, not a pointer to current code. If so, say that in the file's _comment, because nothing there says it today and the neighbouring allowlists use line the opposite way.
  • Raise it if a future triage pass is actually misled by one of these — or if a gate is ever added that reads this file, at which point the drift stops being cosmetic.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    area: ciSubsystem: cibugSomething isn't workingdocumentationImprovements or additions to documentationtriage: validWell-framed; implement as written

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions