Skip to content

Four gates that could not fail: UBSan modes, sanitizer instrumentation, spec-sync, and the forms key vocabulary - #576

Merged
Yaraslaut merged 4 commits into
masterfrom
ci/gate-batch-541-542-554-560
Sep 19, 2026
Merged

Yaraslaut merged 4 commits into
masterfrom
ci/gate-batch-541-542-554-560

Conversation

@Yaraslaut

Copy link
Copy Markdown
Member

Four gates that could not fail, in one branch: the UBSan leg, the sanitizer
instrumentation, the header↔spec sync gate, and the forms DSL's key vocabulary.

Every one of them is mutated below — the change that should have made it red
and did not — and then shown red after the fix. Per AGENTS.md, "Verify rather
than assert": a gate nobody has seen fail is not evidence.

Closes #541
Closes #542
Closes #554
Closes #560


#560 — include/morph/net was exempt from the spec-sync gate

Broken today, measured. The gate read a word list
(subdomains="core journal offline session forms util") and mapped
include/morph/<sub>/ → docs/spec/<sub>/. Running that exact rule over commit
9445bc04 — which changed WsFrameReader to reject ten previously accepted
classes of WebSocket frame and touched no documentation — prints
OLD GATE PASSES. A second sub-domain was missing too, which the issue did not
name: include/morph/render/, documented in docs/spec/forms/forms.md.

Why the naive fix is wrong (as #560 says): there is no docs/spec/net/, so
adding the word net would have demanded a change no correct PR could make —
it would have failed #558, the PR that documented the change properly.

Fixed. The rule moves into scripts/check_spec_sync.sh, a table of
sub-domain → accepted doc paths, driven by a list of changed paths (which is
what makes it testable at all). Two properties beyond the old gate:

  • every directory under include/morph/ must be either mapped or explicitly
    exempt
    — a new sub-domain is now a failure that names itself, rather than
    an exemption by omission, which is exactly how net and render arrived;
  • every mapping must still point at a doc path that exists, so a renamed
    spec cannot leave a sub-domain gated on something unreachable.

Gate proven red, then green (scripts/test_check_spec_sync.sh, 18 cases,
all passing locally):

Case Required Result
9445bc04's path list reject rejected
a72787ed's path list (#558, same change + docs) accept accepted
net header alone / render header alone / core / forms reject rejected
net + docs/spec/core/backend.md, net + docs/spec/security.md accept accepted
qt + detail alone (the two exempt sub-domains) accept accepted
a sub-domain in neither list reject rejected
all of a mapping's doc targets renamed away reject rejected
the sub-domain scan finding nothing (vacuity guard) reject rejected

The two acceptance commits are pinned as their exact git diff --name-only
output, because neither is an ancestor of master (both live on
origin/batch/533-535-536, which a branch deletion would take with it) — and
the self-test reconciles the recorded lists against the commits whenever the
objects are in the checkout, so the fixture cannot rot silently. The self-test
runs in drift-guard.yml.

#554 — the forms DSL's key vocabulary had no completeness gate

Broken today, measured. Check 6 of scripts/check_spec_citations.sh, in the
shape of check 4 (wire.md, #233) with per-category floors, not a lumped
total
— check 4's own comment records why a lumped count let a whole category
parse to nothing while reporting green.

Inventory re-derived mechanically rather than taken from the issue: 27 x-*
keys (not 26), 5 document keys, 16 rule kinds.

Gate proven red, then green — each mutation applied alone in a scratch tree
(scripts/test_check_spec_citations.sh):

Mutation Before After
delete x-placeholder's row from forms.md passed caught
add an undocumented "x-…" literal to a forms header passed caught
delete w-steps's row passed caught
drop RuleKind::AtLeastOneOf from its row passed caught
add a new enumerator to ruleKindName() passed caught
blind the x-* scan (category parses to 0) passed caught
blind the rule-kind scan (category parses to 0) passed caught

Rule kinds are read off ruleKindName()'s switch — the single place mapping a
C++ enumerator to its wire spelling — and each row must now name both. That
closes the asymmetry #554 measured (a JSON author was fully served; a C++ author
had to guess AtLeastOneOf from atLeastOneOf) by construction: forms.md's
kind table gains a RuleKind column.

The six documentation gaps are closed: HasExplicitSubmit and RuleLiteral
where the members they detect are specified, fieldKeyStem/fieldSlotName/
withSlot where the i18n key format is, and groupKindName where
x-layout.groups[].kind is. One correction to the issue: the seventh name
on its list, Plain, is a function-local std::remove_cvref_t alias
(forms.hpp:490 and friends), not a public rule construct — grep 'struct Plain' finds nothing. It is deliberately left undocumented.

#541 — UBSan findings printed and exited 0

The UBSan half already landed in #555 (-fno-sanitize-recover=undefined on
both arms, plus scripts/check_sanitizer_can_fail.sh). Re-measured here rather
than assumed: with that flag stripped from both arms, the probe prints
runtime error: signed integer overflow and exits 0 on ubsan and asan,
and the check reports it. So the gate does go red for the defect it names.

What #555 did not close, and this does, is the same "cannot fail" shape through
the other door, which #541 filed under "Related, same function":
apply_sanitizers() had no else() arm, so -DAF_SANITIZER=msan, =ASAN
or a typo produced a fully uninstrumented build that configured, compiled and
ran the whole suite green under a job named for a sanitizer.

Gate proven red, then green: with the new arm removed, the check reports
"apply_sanitizers() accepted the unknown mode 'msan' and configured anyway" —
which is master's state today. With it, an unknown sanitizer mode fails the configure.

#542 — five suites were never sanitizer-instrumented

morph is header-only, so a target with no apply_sanitizers() block compiles
its own uninstrumented copy of every header it includes.

Gate proven red, then green. On a real clang-asan configure with
-DMORPH_BUILD_QT=ON (local, clang 22.1.8):

$ nm -C build/clang-asan/tests/qt/morph_qt_tests | grep -c __asan_
0
$ bash scripts/check_sanitizer_instrumentation.sh build/clang-asan asan
::error::...morph_qt_tests is run by ctest on the asan leg but carries no __asan_ symbols
::error::check_sanitizer_instrumentation: 1 of 8 ctest binaries are not asan-instrumented

after:

$ bash scripts/check_sanitizer_instrumentation.sh build/clang-asan asan
check_sanitizer_instrumentation: 8 ctest binaries all carry __asan_ symbols (0 allowlisted).

Instrumented: tests/qt (including the two spawned process-separation
helpers — the include/morph/qt code under test only runs in the child),
tests/net_qt_interop, tests/soak, tests/bench, src/qt/forms, plus
morph_qt_impl and examples/qt_tls_client.

morph_qt_impl is the substance: the Qt WebSocket server and backend are the
only compiled TUs in morph's own transport, so a leg that left them out
watched every caller of that code and none of the code. That makes the
qt_tls_client block load-bearing rather than tidy — a consumer of an
instrumented static archive that is not itself instrumented fails to link
(undefined reference to __asan_memset, confirmed), which is the trap
morph_test_main's comment already records and why that one stays exempt.

The assertion is generalised, which is the other half of the ticket.
ladder-sanitizers carried a hand-rolled nm | grep __asan_ over
examples/*/ladder_*_tests — the same blind spot one level up, since that job
also builds tests/qt, examples/qt_tls_client and examples/concepts, none
of which the glob reaches. Both it and kanban-tsan now run the shared script.
kanban-tsan also gains the TSAN_OPTIONS=suppressions=cmake/tsan.supp that
linux-sanitizers has had since #476.

And ladder-sanitizers now runs the Qt transport suites. It is the only leg
that builds Qt under a sanitizer, so instrumenting a suite nothing runs would
have swapped one vacuous control for another. Measured before adding the step:
all 67 cases across tests/qt, tests/net_qt_interop and
qt_tls_example_runs pass clean under -fsanitize=address,undefined with that
step's own options; the -R filter selects the 57 that exist in that job's
configure and carries --no-tests=error, so a renamed TEST_CASE cannot leave
it reporting success having run nothing.

soak/bench were the one judgement call #542 asked to be made explicitly:
instrumented, because they are opt-in (no default leg pays for them) and
churn over thousands of cycles is exactly the shape of test whose finding is a
leak or a race rather than a failed assertion. Recorded in
docs/spec/testing_strategy.md; the two new gates are recorded in
docs/spec/testing_charter.md.


Defects the new instrumentation surfaced

None so far. The 67 newly-instrumented Qt/interop cases pass clean under
ASan+UBSan locally, with and without leak detection on the cases tried. #537
(INT64_MIN UB in Rational) remains the known live UB and has #561 open; it
is not reached by anything this branch newly instruments or newly runs. If CI's
Qt/ladder legs surface anything this local run did not, it gets its own issue
with the real output, per AGENTS.md — not a fix in this PR.

No suppressions added, and no exemption widened. The only exemptions here
are ones that were already there and are now stated: detail and qt in the
spec-sync table, morph_test_main in the instrumentation check.

Why one PR

All four are gate/CI-hygiene changes with no overlapping files except
ci.yml (#541/#542) and check_spec_citations.sh (#554). One CI cycle, one
rebase, one merge.

Review notes (done inline, not by a forked reviewer)

  • Every apply_sanitizers() call site is guarded by if(DEFINED AF_SANITIZER), so the new FATAL_ERROR arm cannot fire on an ordinary
    build; verified across all 20 call sites.
  • Every consumer of the two newly-instrumented static libraries was enumerated
    before instrumenting them (morph_qt_impl: ladder rungs, examples/common,
    examples/bank, tests/qt, tests/net_qt_interop,
    examples/qt_tls_client, and the WASM spike, which never defines
    AF_SANITIZER).
  • linux-all-features builds soak/bench/forms_qml on clang-debug and
    gcc-debug, where AF_SANITIZER is undefined, so those blocks are inert
    there.
  • All three touched workflow files parse (yaml.safe_load);
    check_ci_clang_pin.sh and check_rung_filters.sh still pass, and nothing
    in the tree referenced the removed nm step.
  • The new self-test's scratch tree renames a mapping target to a path outside
    docs/spec/, because check_spec_citations.sh's dangling-reference scan
    reads .sh files too — caught by running that lint over this branch.

🤖 Generated with Claude Code

https://claude.ai/code/session_01DbGrZGkr2WqcAs2DJyMym6

Yaraslaut and others added 4 commits September 18, 2026 22:29
…#560)

The gate read a word list -- `subdomains="core journal offline session forms
util"` -- and mapped `include/morph/<sub>/` to `docs/spec/<sub>/`. Two
sub-domains were missing from it, and a sub-domain missing from that list is
not merely unchecked: it is indistinguishable from one that was deliberately
exempted. Commit 9445bc0 changed WsFrameReader to reject ten previously
accepted classes of WebSocket frame -- an interop-visible change to the
reference transport -- and touched no documentation at all, with every gate
green.

Adding the word `net` does not fix it. There is no docs/spec/net/ folder:
morph::net is documented in docs/spec/core/backend.md and docs/spec/security.md,
and morph::render in docs/spec/forms/forms.md. The naive edit would have
demanded a change no correct pull request could make, and would have failed
morph#558 -- the pull request that documented the change properly.

So the rule moves out of the workflow into scripts/check_spec_sync.sh, where it
reads a table of sub-domain -> accepted doc paths, and where it can be driven
against a list of paths instead of only against a pull_request event. Two
properties beyond the old gate:

- Every directory under include/morph/ must be either mapped or explicitly
  exempt. A new sub-domain is now a failure that names itself rather than an
  exemption by omission, which is how net and render arrived unchecked.
- Every mapping must still point at a doc path that exists, so a renamed spec
  cannot leave a sub-domain gated on something unreachable.

scripts/test_check_spec_sync.sh drives both directions: every rejection the
gate claims (including its own scan going blind), every acceptance it must not
manufacture, and morph#560's two acceptance cases -- 9445bc0's path list must
be rejected, a72787e's must be accepted. Measured before the change: the old
rule accepts 9445bc0's diff.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01DbGrZGkr2WqcAs2DJyMym6
…554)

morph::forms is a domain-specific language: a renderer that is not morph's own
implements 27 `x-*` schema keys, 5 document keys and 16 rule/condition kinds
from docs/spec/forms/, whose table introduces itself as "Renderer contract: the
schema key vocabulary". That is the claim docs/spec/core/wire.md made before
morph#233, and it fails the same way -- morph's own Qt/QML renderer reads the
same headers the schema generator does, so nothing in the product can observe
the spec falling behind the code. Neither existing gate can see completeness:
spec-sync.yml is satisfied by a typo fix under docs/spec/forms/, and checks 1-3
work outward from a hand-maintained pinned-facts list, where a key nobody
pinned is a key nobody checks.

Check 6 of scripts/check_spec_citations.sh reads both sides, in the shape of
check 4 and with per-category floors rather than a lumped total (check 4's own
comment records why: a lumped count let one category parse to nothing while the
others carried it over the line).

Rule kinds are read off ruleKindName()'s switch -- the single place that maps a
C++ enumerator to its wire spelling -- and each row must now name both. That
closes by construction the asymmetry morph#554 measured: every membership and
comparison construct appeared in the spec only under its JSON key spelling, so
a reader emitting JSON was fully served while a reader writing C++ had to guess
the capitalisation of `AtLeastOneOf` from `atLeastOneOf`. forms.md's kind table
gains a `RuleKind` column accordingly.

Measured, per category, each mutation applied alone in a scratch tree:
deleting `x-placeholder`'s row, adding an undocumented `"x-…"` literal to a
header, deleting `w-steps`'s row, dropping `RuleKind::AtLeastOneOf` from its
row, adding a new enumerator to ruleKindName(), and blinding either scan so the
category parses to zero -- all six are now rejected, and all six passed before.
scripts/test_check_spec_citations.sh carries them.

Also closes the six documentation gaps morph#554 listed: `HasExplicitSubmit`
and `RuleLiteral` are named where the members they detect are specified,
`fieldKeyStem`/`fieldSlotName`/`withSlot` where the i18n key format is, and
`groupKindName` where `x-layout.groups[].kind` is. The seventh name on that
list, `Plain`, is a function-local `std::remove_cvref_t` alias, not a public
rule construct, and is deliberately left undocumented.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01DbGrZGkr2WqcAs2DJyMym6
…nts (#541)

The UBSan half of morph#541 landed in morph#555: apply_sanitizers() passes
-fno-sanitize-recover=undefined on both the asan and ubsan arms, and
scripts/check_sanitizer_can_fail.sh drives the function behaviourally rather
than grepping for the flag. Re-measured here before touching anything: with
that flag removed from both arms, the probe prints `runtime error: signed
integer overflow` and exits 0 on each -- so the gate does go red for the defect
it names, and is green today because the flag is really on the compile line.

What morph#541 also reported, and #555 did not close, is the same "cannot fail"
shape arriving through the other door. apply_sanitizers() was `if asan / elseif
tsan / elseif ubsan / endif` with no else arm, so `-DAF_SANITIZER=msan`, `=ASAN`
or a typo silently produced a completely uninstrumented build that configured,
compiled and ran the whole suite green, under a job named for a sanitizer.

Now it is a configure-time FATAL_ERROR, and check_sanitizer_can_fail.sh asserts
that behaviourally too: a probe configured with mode `msan` must fail, and fail
with apply_sanitizers()'s own diagnostic rather than by accident. Measured with
the new arm removed again -- the check reports that the unknown mode configured
anyway, which is the state master is in today.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01DbGrZGkr2WqcAs2DJyMym6
morph is header-only, so a target with no apply_sanitizers() block compiles its
own uninstrumented copy of every header it includes. Five suites had none:
tests/qt, tests/net_qt_interop, tests/soak, tests/bench and src/qt/forms.
(tests/offline_sqlite, the sharp case morph#542 opened with, was fixed in
morph#555.)

Measured, on a clang-asan configure with -DMORPH_BUILD_QT=ON: morph_qt_tests
links with zero `__asan_` symbols, and
scripts/check_sanitizer_instrumentation.sh reports it -- "1 of 8 ctest binaries
are not asan-instrumented". With the blocks added, all 8 carry them.

morph_qt_impl is instrumented too: the Qt WebSocket server and backend are the
only *compiled* translation units in morph's own transport, so a leg that left
them out watched every caller of that code and none of the code. That makes
examples/qt_tls_client's block load-bearing rather than tidy -- a consumer of
an instrumented static archive that is not itself instrumented fails to link
(confirmed: `undefined reference to __asan_memset`), which is the failure mode
morph_test_main's comment in the root CMakeLists.txt records, and is why that
one stays exempt.

The instrumentation assertion is generalised, which is the other half of the
ticket. ladder-sanitizers carried a hand-rolled `nm | grep __asan_` over
`examples/*/ladder_*_tests` -- the same blind spot one level up, since that job
also builds tests/qt, examples/qt_tls_client and examples/concepts, none of
which the glob reaches. Both it and kanban-tsan now run the shared script,
which walks what ctest will actually run and keys the expected symbol on the
leg. kanban-tsan also gains the `TSAN_OPTIONS=suppressions=cmake/tsan.supp`
that linux-sanitizers has had since morph#476; without it that leg is a coin
toss on a false positive the other TSan leg already knows is one.

And ladder-sanitizers now *runs* the Qt transport suites. It is the only leg
that builds Qt under a sanitizer, so instrumenting a suite nothing runs would
have swapped one vacuous control for another. Measured before adding the step:
all 67 cases across tests/qt, tests/net_qt_interop and qt_tls_example_runs pass
clean under -fsanitize=address,undefined with that step's own options, on
clang 22; the step's `-R` filter selects the 57 that exist in that job's
configure, and carries `--no-tests=error` so a renamed TEST_CASE cannot leave
it reporting success having run nothing.

soak/bench were the one judgement call morph#542 asked to be made explicitly:
instrumented, because they are opt-in (no default leg pays for them) and churn
over thousands of cycles is exactly the shape of test whose finding is a leak
or a race rather than a failed assertion. docs/spec/testing_strategy.md and
docs/spec/testing_charter.md record that decision and the two new gates.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01DbGrZGkr2WqcAs2DJyMym6
@codecov

codecov Bot commented Sep 18, 2026

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.

📢 Thoughts on this report? Let us know!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment