Framework under-assertion: what the coverage + mutation round actually left, after #434

[Yaraslaut]

What this replaces

Four issues — #409 (offline coverage), #410 (forms coverage), #431 (core survivors), #432 (forms survivors) — were split out of one measurement round on 2026-09-03. Each has since had its own acceptance criteria withdrawn, by the same two events, and each carries a comment saying so. This issue is the surviving work; each original stays readable at its link with its evidence and its triage history intact.

The two events.

1. The percentage clauses were superseded by their own authors (2026-09-03, identical comments on coverage: include/morph/offline is the library's worst subsystem at 85.20%, and one header inside it reads 57.04% #409 and coverage: include/morph/forms sits at 94.37%, and forms.hpp alone holds 36 of the subsystem's 41 unhit lines #410). The first mutation run scored 64.06% against a library measuring 95.69% lines — the gap between executed and would be noticed if it were wrong. A percentage target is what produced a suite on the wrong side of it.
2. The survivor lists were invalidated by mutation: a Mull survivor is not evidence of a missing assertion — four sampled survivors all turn the suite red when applied by hand #434 (mechanism confirmed 2026-09-04, landed in PR mutation: confirm the false-positive mechanism -- cxx_remove_void_call is broken for member-call sites #448). Mull 0.34.0's cxx_remove_void_call does not remove the call it claims to, for member/operator call sites — confirmed by disassembling the mutant in a single-TU build. That is 148 of the 352 survivors, and it is the shape of remote.hpp's reply(...)/complete(...) and bridge.hpp's stored-callback sites — 211 of the 352 between them.

Both "start here" sections are therefore closed: #431's twenty deletable replies are tool artifacts, and #431/#432's named boundary sites are already killed (scripts/mutation_survivors.json, runs[1].movement_against_baseline).

What is actually left

A. The survivors from mutator families #434 does not implicate. runs[1].survivors_by_mutator: cxx_replace_scalar_call 97, cxx_init_const 26, cxx_assign_const 23, cxx_add_to_sub 13, cxx_gt_to_ge 8, other 37 — ~204 of the 352. A boundary-comparison mutator was independently confirmed in #434's investigation to mutate and kill correctly, so this family's survivors should still be read as real. Classify each: killed, equivalent with a reason, or side-channel. None left unclassified.

B. The shape, checked by reading rather than by tooling. "A test whose name claims a boundary and whose body tests the middle" was found in wire.hpp by reading the test, not by trusting a survivor — the case named "decode accepts an envelope at the size limit boundary" asserted a 1 KiB payload against an 8 MiB cap. The forms boundaries deserve the same read; it needs no mutation tooling and is unblocked today.

C. offline's coverage, re-measured now that the denominator is fixed. #409's blockers are both closed (#403 coverage.sh profiling, #440 the SQLite3 IMPORTED_LOCATION fix), and its own text says the honest first step is to re-measure rather than to start writing tests. sqlite_offline_queue.hpp's 57.04% was measured with its own test binary unprofiled; record both the before and after numbers, so it is visible how much of the gap was reporting and how much was tests. If it barely moves, morph_offline_sqlite_tests is thin, which is a worse finding than the percentage suggested.

D. The codecov components neither subsystem has. codecov.yml today carries framework, ladder and one component per rung. Neither include/morph/offline nor include/morph/forms has one, so neither can regress into a red status. Add them with targets derived from measured ceilings, after C.

The constraint that carries over unchanged

Do not chase the number. A mutant killed by an assertion written to kill that mutant, on a path nobody cares about, is worse than a survivor — it is a test with no reason to exist. Each new test must be justifiable from the behaviour, with the survivor as evidence the assertion was missing. Classifying a survivor equivalent with a stated reason is a valid outcome; the run already found 11.

Not in scope

#434 (the Mull defect itself, still open with an upstream filing outstanding) and #408 (the CI gate) stay separate.

[Yaraslaut]

Triaged: triage: valid, area: ci — with one qualification: this is a
program of four items, not a change, and it must be split before anyone
opens a PR against it. See "How to pick this up" at the end.

Why valid

The usual reason a measurement issue gets rescoped here is that it carries a
percentage target, and this one has already had both of its bad framings
removed by its own authors rather than by a triager: the percentage clauses
were withdrawn on #409/#410, and the survivor lists were invalidated by #434.
What is left is stated as behaviour ("classify each survivor: killed,
equivalent with a reason, or side-channel"), not as a number to reach, and
the closing constraint — "a mutant killed by an assertion written to kill that
mutant, on a path nobody cares about, is worse than a survivor" — is skill
invariant 7 ("would this still pass if the feature did nothing?") stated in the
issue's own words. That is the opposite of the failure mode.

What I verified

The replaced issues really are closed, so this is not a duplicate queue.
#409, #410, #431, #432 — all CLOSED. The two blockers item C names as
cleared, #403 and #440, are both CLOSED too, so C's "re-measure first" step
is genuinely unblocked today. #408 and #434, named as out of scope, are both
still OPEN — the scope boundary the issue draws is accurate.

Item D is confirmed against origin/master. codecov.yml carries
component_ids framework, ladder, pastebin, bookmarks, polls,
kanban, ledger, lims, crm — and nothing for include/morph/offline or
include/morph/forms. Neither subsystem can produce a red status, exactly as
claimed. Note the file already documents one prior instance of this same hole
("examples/rungs.txt has no component_id here… so the omission was
unguarded"), which is corroboration that the gate cannot see what it has no
component for.

Item B's own worked example has already landed, and it is the template.
tests/test_wire_hardening.cpp:79 still holds the case the issue quotes —
named "decode accepts an envelope at the size limit boundary", body asserting
std::string(1024, 'x') against kMaxEnvelopeBytes:

TEST_CASE("decode accepts an envelope at the size limit boundary", "[wire][hardening]") {
    // Build a valid envelope whose serialized form is comfortably under the cap.
    ...
    env.body = std::string(1024, 'x');  // small, valid string payload

— but it is now followed on master by an explicit comment naming the defect
("The boundary the case above is named for, and does not actually reach…
mutating > to >= … passed the entire suite") and the real boundary cases.
So B is not speculative: the shape was found by reading, the fix shape exists
in the tree, and what remains is applying the same read to the forms
boundaries. That is the cheapest and most clearly-specified item here.

How to pick this up

Do not batch this into a multi-issue fix PR; A alone is ~204 survivors to
classify. Split it, in this order:

1. B — forms boundary tests (unblocked, bounded, no tooling needed). File
   as its own issue; tests/test_wire_hardening.cpp:79-100 on master is the
   worked precedent to copy, comment and all.
2. C — re-measure include/morph/offline coverage (unblocked; both
   blockers closed). Record the before and after numbers, per the issue.
3. D — the two missing codecov.yml components, with targets derived from
   C's measured ceilings. Depends on C; the file's existing rung components
   show the required shape, including the measured-ceiling comment convention.
4. A — classify the ~204 non-cxx_remove_void_call survivors. Largest, and
   the only one that needs scripts/mutation.sh re-run.

Leave this issue open as the parent while those land.

[Yaraslaut]

What PR #476 closes here, and what is left

#476 now carries C and D. B is not done — I checked by reading rather than assuming, and it is worth being precise about why. A is untouched and is the only item that survives, so this issue stays open.

---

D — the two missing components. Done (3f3af032).

codecov.yml now carries component_id: offline (include/morph/offline/**, target 95%) and component_id: forms (include/morph/forms/**, target 97%), both informational: true, both following the file's existing measured-ceiling comment convention.

The premise held on inspection: before this, framework scored all of include/morph as one number, and include/morph/offline is 639 of its 8,645 lines — so offline could fall from 96% to 60%, move the whole-library figure by about two and a half points, and name no subsystem while doing it. That is the same hole the file already documents one level down for the rungs ("check_rung_filters.sh checks that each rung has a component_id, not that it has these, so the omission was unguarded").

Targets are a point under each measured ceiling, the margin every other target in the file leaves. scripts/check_rung_filters.sh passes (39 checks), and the file validates against Codecov's own endpoint with all 11 components resolving to the intended paths and targets — worth checking explicitly, since a file that fails validation is discarded wholesale and Codecov's blocking defaults silently replace it (morph#133).

C — re-measure offline. Done, and I re-derived it rather than taking it on trust.

#476's framework comment reports the post-work subsystem figures. I ran CI's own clang-coverage configure locally (NET/OFFLINE_SQLITE/QT/LADDER=ON, rungs all; 2,706 tests, 100% passing) and applied Codecov's arithmetic to the uploaded coverage.lcov:

offline    96.56%    639 lines    11 misses,  11 partials     <- digit for digit
forms      98.60%  1,577 lines    10 misses,  12 partials     <- vs 98.43% claimed

Forms differs only because #476's run had MORPH_BUILD_LADDER=OFF, which its own comment already flags as making its numbers a lower bound.

This issue asked for the before and after of the one header the subsystem figure was hiding, not just the subsystem, so:

include/morph/offline/sqlite_offline_queue.hpp
    before (#409)   57.04%    52 misses, 12 partials
    after           91.53%   189 lines,  8 misses, 8 partials

Both numbers are now recorded in codecov.yml next to the new components. The answer to this issue's own question — "if it barely moves, morph_offline_sqlite_tests is thin, which is a worse finding" — is that most of the move is reporting (#403: the binary was never handed to llvm-cov, so its hits were dropped), but not all of it: #476 still closed six genuine gaps in that header and documented two it could not reach. The suite was not thin, it was invisible.

B — the forms boundary read. Not done. Reporting this rather than claiming it.

#476 touches test_forms_field_bounds.cpp, test_forms_instance_constraints.cpp, test_forms_rules.cpp and test_quantity_forms.cpp, and forms did reach 98.43%. But the diff across all four is pure addition — 11 new TEST_CASEs, zero modifications and zero removals. The defect this item names is "a test whose name claims a boundary and whose body tests the middle", and fixing one means editing an existing test. That count is zero by construction. The plan doc corroborates it: Phase 3 is scoped to closing forms's line/branch gaps, and the cxx_gt_to_ge survivors — precisely the mutator that catches this shape — are listed among the deferred work.

The shape is still there. Three concrete instances:

1. tests/test_forms_exact_bounds.cpp:80 — "schemaJson leaves bounds a double holds exactly untouched". The limit it is named for is kExactDoubleLimit = 9007199254740992 (2⁵³, forms.hpp:1047), enforced with strict >/< at forms.hpp:2418/:2423. The body uses a type whose bounds are ±2³¹. Four million times short of the limit in the name — this is the wire.hpp case again, almost exactly. Its siblings at :68/:75 use INT64_MAX/UINT64_MAX, far past the limit, so neither side of the edge is pinned and cmp_greater → cmp_greater_equal survives the whole suite. No bound of exactly 9007199254740992 is declared anywhere in the tree.
2. instance_constraints.hpp:316/:319 — checkValue's min/max. The doc calls the range inclusive; every asserting value in the suite is well inside or well outside (40 against a max of 10, −5 against a min of 0; the shared corpus declares 0/20 and tests 12, 12.5, 25, −1). Nothing asserts a value equal to a declared bound, so < → <= and > → >= both survive. The names here are honest ("below", "out-of-range"), so this is a missing boundary test rather than a mislabelled one — but it is the same hole, and Framework coverage & mutation improvement plan #476's four new cases in that very file did not touch it.
3. forms.hpp:1114, Equals::emitNode — same 2⁵³ threshold. Both sides are pinned one past the limit (test_forms_rule_agreement.cpp:45 and Framework coverage & mutation improvement plan #476's own test_forms_rules.cpp:585), neither side has the "at the limit, nothing emitted" half.

For the record: nothing in the tree carries an in-tree note that this read was done for forms. grep for the wire.hpp comment's phrasing across tests/, include/ and docs/ returns exactly one hit — tests/test_wire_hardening.cpp:88, the original.

This is a small, well-specified piece of work with three named starting points and a worked precedent to copy. It is not something to bolt onto a 62-file PR, so I have left it.

A — classify the ~204 survivors. Untouched, out of scope.

Needs a full scripts/mutation.sh re-run (~46 min), and its value still depends on #408's undecided question of whether the campaign runs in CI at all. Deliberately not attempted.

---

So: C and D close with #476; B and A remain, and this issue stays open for them. If you want them split as the triage comment suggests, B is now the cheaper of the two and has three concrete sites to start from.

🤖 Generated with Claude Code

https://claude.ai/code/session_01C6uB9zxdSG8qp3VNAAyFvF

[Yaraslaut]

Decision: this issue is now item A alone; B is split out, C and D are closed

C — re-measure offline coverage: CLOSED by PR #476. It ran the full
audit-close-remeasure pass and recorded the per-header before/after this issue
asked for: sqlite_offline_queue.hpp 57.04% → 91.53%, offline overall
96.56% (639 lines, 11 misses, 11 partials). The 57.04% premise — measured with
that binary unprofiled — is gone.

D — codecov components for offline and forms: CLOSED by PR #476. Both
components were added, with targets re-derived from the measured numbers rather
than copied. This was the item that protects everything #476 achieved: without
it neither subsystem could regress into a red status.

B — read the forms boundary tests: SPLIT OUT as morph#484. Honestly reported
as not done: #476 touched those files by addition only (11 new cases, zero
modifications), so the "name claims a boundary, body tests the middle" shape is
untouched by construction. Three instances are named there, the clearest being
test_forms_exact_bounds.cpp:80 asserting ±2³¹ against a real limit of 2⁵³. It
is small and unblocked, which is why it should not stay buried in a four-item
epic.

A — classify the ~204 non-cxx_remove_void_call survivors: REMAINS, and is
now sequenced. morph#408 is decided: the mutation campaign runs on a
schedule, not per-PR, with cxx_remove_void_call excluded from the outset.
That changes A from a prerequisite into work against the scheduled run's output
— classify each survivor as killed, equivalent with a stated reason, or side
channel, from a run whose mutator set is no longer 42% artefact.

So A is blocked on #408 landing, and no earlier. Re-running the campaign by hand
first would produce a list that the scheduled job's own configuration then
invalidates.

The constraint that still governs

Unchanged, and worth restating because A is the item most exposed to it:

Do not chase the number. A mutant killed by an assertion written to kill that
mutant, on a path nobody cares about, is worse than a survivor — it is a test
with no reason to exist.

Classifying a survivor equivalent with a stated reason remains a valid
outcome; the previous run already found 11.

This issue stays open, scoped to A, blocked on morph#408.

[Yaraslaut]

Item A is unblocked: the scheduled campaign has produced a baseline

morph#408 is closed. Its workflow ran end-to-end on GitHub's infrastructure (run 34349442137, ✓ 2h25m18s) and committed the first baseline measured under the reduced mutator set:

"core-forms": { "survived": 199, "mutants": 773 }

199 survivors, 773 mutants, 74.3% mutation score — with cxx_remove_void_call excluded from scripts/mutation.sh's mutator list before execution, so the population is no longer 42% tool artefact. The report is attached to that run as mutation-report-core-forms.

This is the condition this issue set for item A: "classify each survivor as killed, equivalent with a stated reason, or side channel, from a run whose mutator set is no longer 42% artefact" — and "blocked on #408 landing, and no earlier". It has landed.

Two notes for whoever picks A up:

• 199, not the ~204 this issue estimated. The estimate came from subtracting cxx_remove_void_call's 148 from the old 352-survivor run. The new figure is a fresh measurement over a different mutant population, not that subtraction — so treat 199 as the real starting list and the ~204 as superseded.
• The constraint still governs. "A mutant killed by an assertion written to kill that mutant, on a path nobody cares about, is worse than a survivor." Classifying a survivor equivalent with a stated reason remains a valid outcome; the previous run already found 11.

[Yaraslaut]

Item A: run recorded, 16 of 199 classified, and one real gap closed — PR #509

Not a finished pass, and the recorded file says so rather than implying one.

Recorded run[2] in scripts/mutation_survivors.json from the first scheduled-gate campaign (run 34349442137): 773 mutants, 574 killed, 199 survived, 74.26%, with per-file and per-mutator breakdowns. Deliberately marked as not comparable to the 352-survivor figure above it — a different, larger population, before cxx_remove_void_call was excluded.

Method. Clustered by (mutator, source shape) rather than walking 199 mutants one at a time. That put sixteen bitwise survivors in one bucket, and they split two ways.

Thirteen are provably equivalent, with the reason already in the source:

• remote.hpp:157-162 — inside OpaqueIdGenerator::mix. The Feistel construction (L,R) -> (R, L ^ F(R)) is invertible for any round function, which permute()'s own doc comment states. Mutating F preserves the only property the code claims.
• bridge.hpp:121, registry.hpp:54, payload_schema.hpp:218 — hash-combine sites. Nothing in the contract fixes a hash value, so killing these would mean asserting a hash constant. That is precisely the "mutant killed by an assertion written to kill that mutant, on a path nobody cares about" this issue's own constraint rules out.

Three were a genuine hole, and it is a good one. They sit on the Feistel structure, not inside it, and they survive for a reason worth stating plainly: the existing bijection test drives counters 1..20000, every one of which has a zero high half. It therefore passes whether or not permute() uses the top 32 bits at all — invariant 7, exactly.

Simulated against the real round function over 20000 counters that do exercise the high half:

mutation	distinct ids (of 20000)
baseline	20000
counter >> 32 becomes <<	1
Feistel ^ becomes |	1256
hi << 32 becomes >>	20000

Catastrophic collisions in opaque model ids — which exist so a client cannot guess another client's id — and completely invisible to the existing sample. The new test strides counters by 2^32 and kills the first two. The third still survives (hi >> 32 yields 0, so the id degenerates to its low half, which stays distinct for these inputs); killing it needs a probe on the id's high bits rather than a distinctness count, and that is recorded rather than glossed over.

183 of 199 remain unclassified. remote.hpp (82) and bridge.hpp (33) hold 58% of them and have not been assessed; the largest cluster is cxx_replace_scalar_call (92), concentrated on .empty()/.size() guards. This issue stays open for those.