Four gates that could not fail: UBSan modes, sanitizer instrumentation, spec-sync, and the forms key vocabulary - #576
Merged
Conversation
…#560) The gate read a word list -- `subdomains="core journal offline session forms util"` -- and mapped `include/morph/<sub>/` to `docs/spec/<sub>/`. Two sub-domains were missing from it, and a sub-domain missing from that list is not merely unchecked: it is indistinguishable from one that was deliberately exempted. Commit 9445bc0 changed WsFrameReader to reject ten previously accepted classes of WebSocket frame -- an interop-visible change to the reference transport -- and touched no documentation at all, with every gate green. Adding the word `net` does not fix it. There is no docs/spec/net/ folder: morph::net is documented in docs/spec/core/backend.md and docs/spec/security.md, and morph::render in docs/spec/forms/forms.md. The naive edit would have demanded a change no correct pull request could make, and would have failed morph#558 -- the pull request that documented the change properly. So the rule moves out of the workflow into scripts/check_spec_sync.sh, where it reads a table of sub-domain -> accepted doc paths, and where it can be driven against a list of paths instead of only against a pull_request event. Two properties beyond the old gate: - Every directory under include/morph/ must be either mapped or explicitly exempt. A new sub-domain is now a failure that names itself rather than an exemption by omission, which is how net and render arrived unchecked. - Every mapping must still point at a doc path that exists, so a renamed spec cannot leave a sub-domain gated on something unreachable. scripts/test_check_spec_sync.sh drives both directions: every rejection the gate claims (including its own scan going blind), every acceptance it must not manufacture, and morph#560's two acceptance cases -- 9445bc0's path list must be rejected, a72787e's must be accepted. Measured before the change: the old rule accepts 9445bc0's diff. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01DbGrZGkr2WqcAs2DJyMym6
…554) morph::forms is a domain-specific language: a renderer that is not morph's own implements 27 `x-*` schema keys, 5 document keys and 16 rule/condition kinds from docs/spec/forms/, whose table introduces itself as "Renderer contract: the schema key vocabulary". That is the claim docs/spec/core/wire.md made before morph#233, and it fails the same way -- morph's own Qt/QML renderer reads the same headers the schema generator does, so nothing in the product can observe the spec falling behind the code. Neither existing gate can see completeness: spec-sync.yml is satisfied by a typo fix under docs/spec/forms/, and checks 1-3 work outward from a hand-maintained pinned-facts list, where a key nobody pinned is a key nobody checks. Check 6 of scripts/check_spec_citations.sh reads both sides, in the shape of check 4 and with per-category floors rather than a lumped total (check 4's own comment records why: a lumped count let one category parse to nothing while the others carried it over the line). Rule kinds are read off ruleKindName()'s switch -- the single place that maps a C++ enumerator to its wire spelling -- and each row must now name both. That closes by construction the asymmetry morph#554 measured: every membership and comparison construct appeared in the spec only under its JSON key spelling, so a reader emitting JSON was fully served while a reader writing C++ had to guess the capitalisation of `AtLeastOneOf` from `atLeastOneOf`. forms.md's kind table gains a `RuleKind` column accordingly. Measured, per category, each mutation applied alone in a scratch tree: deleting `x-placeholder`'s row, adding an undocumented `"x-…"` literal to a header, deleting `w-steps`'s row, dropping `RuleKind::AtLeastOneOf` from its row, adding a new enumerator to ruleKindName(), and blinding either scan so the category parses to zero -- all six are now rejected, and all six passed before. scripts/test_check_spec_citations.sh carries them. Also closes the six documentation gaps morph#554 listed: `HasExplicitSubmit` and `RuleLiteral` are named where the members they detect are specified, `fieldKeyStem`/`fieldSlotName`/`withSlot` where the i18n key format is, and `groupKindName` where `x-layout.groups[].kind` is. The seventh name on that list, `Plain`, is a function-local `std::remove_cvref_t` alias, not a public rule construct, and is deliberately left undocumented. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01DbGrZGkr2WqcAs2DJyMym6
…nts (#541) The UBSan half of morph#541 landed in morph#555: apply_sanitizers() passes -fno-sanitize-recover=undefined on both the asan and ubsan arms, and scripts/check_sanitizer_can_fail.sh drives the function behaviourally rather than grepping for the flag. Re-measured here before touching anything: with that flag removed from both arms, the probe prints `runtime error: signed integer overflow` and exits 0 on each -- so the gate does go red for the defect it names, and is green today because the flag is really on the compile line. What morph#541 also reported, and #555 did not close, is the same "cannot fail" shape arriving through the other door. apply_sanitizers() was `if asan / elseif tsan / elseif ubsan / endif` with no else arm, so `-DAF_SANITIZER=msan`, `=ASAN` or a typo silently produced a completely uninstrumented build that configured, compiled and ran the whole suite green, under a job named for a sanitizer. Now it is a configure-time FATAL_ERROR, and check_sanitizer_can_fail.sh asserts that behaviourally too: a probe configured with mode `msan` must fail, and fail with apply_sanitizers()'s own diagnostic rather than by accident. Measured with the new arm removed again -- the check reports that the unknown mode configured anyway, which is the state master is in today. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01DbGrZGkr2WqcAs2DJyMym6
morph is header-only, so a target with no apply_sanitizers() block compiles its own uninstrumented copy of every header it includes. Five suites had none: tests/qt, tests/net_qt_interop, tests/soak, tests/bench and src/qt/forms. (tests/offline_sqlite, the sharp case morph#542 opened with, was fixed in morph#555.) Measured, on a clang-asan configure with -DMORPH_BUILD_QT=ON: morph_qt_tests links with zero `__asan_` symbols, and scripts/check_sanitizer_instrumentation.sh reports it -- "1 of 8 ctest binaries are not asan-instrumented". With the blocks added, all 8 carry them. morph_qt_impl is instrumented too: the Qt WebSocket server and backend are the only *compiled* translation units in morph's own transport, so a leg that left them out watched every caller of that code and none of the code. That makes examples/qt_tls_client's block load-bearing rather than tidy -- a consumer of an instrumented static archive that is not itself instrumented fails to link (confirmed: `undefined reference to __asan_memset`), which is the failure mode morph_test_main's comment in the root CMakeLists.txt records, and is why that one stays exempt. The instrumentation assertion is generalised, which is the other half of the ticket. ladder-sanitizers carried a hand-rolled `nm | grep __asan_` over `examples/*/ladder_*_tests` -- the same blind spot one level up, since that job also builds tests/qt, examples/qt_tls_client and examples/concepts, none of which the glob reaches. Both it and kanban-tsan now run the shared script, which walks what ctest will actually run and keys the expected symbol on the leg. kanban-tsan also gains the `TSAN_OPTIONS=suppressions=cmake/tsan.supp` that linux-sanitizers has had since morph#476; without it that leg is a coin toss on a false positive the other TSan leg already knows is one. And ladder-sanitizers now *runs* the Qt transport suites. It is the only leg that builds Qt under a sanitizer, so instrumenting a suite nothing runs would have swapped one vacuous control for another. Measured before adding the step: all 67 cases across tests/qt, tests/net_qt_interop and qt_tls_example_runs pass clean under -fsanitize=address,undefined with that step's own options, on clang 22; the step's `-R` filter selects the 57 that exist in that job's configure, and carries `--no-tests=error` so a renamed TEST_CASE cannot leave it reporting success having run nothing. soak/bench were the one judgement call morph#542 asked to be made explicitly: instrumented, because they are opt-in (no default leg pays for them) and churn over thousands of cycles is exactly the shape of test whose finding is a leak or a race rather than a failed assertion. docs/spec/testing_strategy.md and docs/spec/testing_charter.md record that decision and the two new gates. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01DbGrZGkr2WqcAs2DJyMym6
Codecov Report✅ All modified and coverable lines are covered by tests. 📢 Thoughts on this report? Let us know! |
This was referenced Sep 19, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Four gates that could not fail, in one branch: the UBSan leg, the sanitizer
instrumentation, the header↔spec sync gate, and the forms DSL's key vocabulary.
Every one of them is mutated below — the change that should have made it red
and did not — and then shown red after the fix. Per
AGENTS.md, "Verify ratherthan assert": a gate nobody has seen fail is not evidence.
Closes #541
Closes #542
Closes #554
Closes #560
#560 —
include/morph/netwas exempt from the spec-sync gateBroken today, measured. The gate read a word list
(
subdomains="core journal offline session forms util") and mappedinclude/morph/<sub>/→docs/spec/<sub>/. Running that exact rule over commit9445bc04— which changedWsFrameReaderto reject ten previously acceptedclasses of WebSocket frame and touched no documentation — prints
OLD GATE PASSES. A second sub-domain was missing too, which the issue did notname:
include/morph/render/, documented indocs/spec/forms/forms.md.Why the naive fix is wrong (as #560 says): there is no
docs/spec/net/, soadding the word
netwould have demanded a change no correct PR could make —it would have failed #558, the PR that documented the change properly.
Fixed. The rule moves into
scripts/check_spec_sync.sh, a table ofsub-domain → accepted doc paths, driven by a list of changed paths (which is
what makes it testable at all). Two properties beyond the old gate:
include/morph/must be either mapped or explicitlyexempt — a new sub-domain is now a failure that names itself, rather than
an exemption by omission, which is exactly how
netandrenderarrived;spec cannot leave a sub-domain gated on something unreachable.
Gate proven red, then green (
scripts/test_check_spec_sync.sh, 18 cases,all passing locally):
9445bc04's path lista72787ed's path list (#558, same change + docs)docs/spec/core/backend.md, net +docs/spec/security.mdqt+detailalone (the two exempt sub-domains)The two acceptance commits are pinned as their exact
git diff --name-onlyoutput, because neither is an ancestor of
master(both live onorigin/batch/533-535-536, which a branch deletion would take with it) — andthe self-test reconciles the recorded lists against the commits whenever the
objects are in the checkout, so the fixture cannot rot silently. The self-test
runs in
drift-guard.yml.#554 — the forms DSL's key vocabulary had no completeness gate
Broken today, measured. Check 6 of
scripts/check_spec_citations.sh, in theshape of check 4 (
wire.md, #233) with per-category floors, not a lumpedtotal — check 4's own comment records why a lumped count let a whole category
parse to nothing while reporting green.
Inventory re-derived mechanically rather than taken from the issue: 27
x-*keys (not 26), 5 document keys, 16 rule kinds.
Gate proven red, then green — each mutation applied alone in a scratch tree
(
scripts/test_check_spec_citations.sh):x-placeholder's row fromforms.md"x-…"literal to a forms headerw-steps's rowRuleKind::AtLeastOneOffrom its rowruleKindName()x-*scan (category parses to 0)Rule kinds are read off
ruleKindName()'s switch — the single place mapping aC++ enumerator to its wire spelling — and each row must now name both. That
closes the asymmetry #554 measured (a JSON author was fully served; a C++ author
had to guess
AtLeastOneOffromatLeastOneOf) by construction:forms.md'skind table gains a
RuleKindcolumn.The six documentation gaps are closed:
HasExplicitSubmitandRuleLiteralwhere the members they detect are specified,
fieldKeyStem/fieldSlotName/withSlotwhere the i18n key format is, andgroupKindNamewherex-layout.groups[].kindis. One correction to the issue: the seventh nameon its list,
Plain, is a function-localstd::remove_cvref_talias(
forms.hpp:490and friends), not a public rule construct —grep 'struct Plain'finds nothing. It is deliberately left undocumented.#541 — UBSan findings printed and exited 0
The UBSan half already landed in #555 (
-fno-sanitize-recover=undefinedonboth arms, plus
scripts/check_sanitizer_can_fail.sh). Re-measured here ratherthan assumed: with that flag stripped from both arms, the probe prints
runtime error: signed integer overflowand exits 0 on ubsan and asan,and the check reports it. So the gate does go red for the defect it names.
What #555 did not close, and this does, is the same "cannot fail" shape through
the other door, which #541 filed under "Related, same function":
apply_sanitizers()had noelse()arm, so-DAF_SANITIZER=msan,=ASANor a typo produced a fully uninstrumented build that configured, compiled and
ran the whole suite green under a job named for a sanitizer.
Gate proven red, then green: with the new arm removed, the check reports
"apply_sanitizers() accepted the unknown mode 'msan' and configured anyway" —
which is master's state today. With it,
an unknown sanitizer mode fails the configure.#542 — five suites were never sanitizer-instrumented
morph is header-only, so a target with no
apply_sanitizers()block compilesits own uninstrumented copy of every header it includes.
Gate proven red, then green. On a real
clang-asanconfigure with-DMORPH_BUILD_QT=ON(local, clang 22.1.8):after:
Instrumented:
tests/qt(including the two spawned process-separationhelpers — the
include/morph/qtcode under test only runs in the child),tests/net_qt_interop,tests/soak,tests/bench,src/qt/forms, plusmorph_qt_implandexamples/qt_tls_client.morph_qt_implis the substance: the Qt WebSocket server and backend are theonly compiled TUs in morph's own transport, so a leg that left them out
watched every caller of that code and none of the code. That makes the
qt_tls_clientblock load-bearing rather than tidy — a consumer of aninstrumented static archive that is not itself instrumented fails to link
(
undefined reference to __asan_memset, confirmed), which is the trapmorph_test_main's comment already records and why that one stays exempt.The assertion is generalised, which is the other half of the ticket.
ladder-sanitizerscarried a hand-rollednm | grep __asan_overexamples/*/ladder_*_tests— the same blind spot one level up, since that jobalso builds
tests/qt,examples/qt_tls_clientandexamples/concepts, noneof which the glob reaches. Both it and
kanban-tsannow run the shared script.kanban-tsanalso gains theTSAN_OPTIONS=suppressions=cmake/tsan.suppthatlinux-sanitizershas had since #476.And
ladder-sanitizersnow runs the Qt transport suites. It is the only legthat builds Qt under a sanitizer, so instrumenting a suite nothing runs would
have swapped one vacuous control for another. Measured before adding the step:
all 67 cases across
tests/qt,tests/net_qt_interopandqt_tls_example_runspass clean under-fsanitize=address,undefinedwith thatstep's own options; the
-Rfilter selects the 57 that exist in that job'sconfigure and carries
--no-tests=error, so a renamedTEST_CASEcannot leaveit reporting success having run nothing.
soak/benchwere the one judgement call #542 asked to be made explicitly:instrumented, because they are opt-in (no default leg pays for them) and
churn over thousands of cycles is exactly the shape of test whose finding is a
leak or a race rather than a failed assertion. Recorded in
docs/spec/testing_strategy.md; the two new gates are recorded indocs/spec/testing_charter.md.Defects the new instrumentation surfaced
None so far. The 67 newly-instrumented Qt/interop cases pass clean under
ASan+UBSan locally, with and without leak detection on the cases tried. #537
(
INT64_MINUB inRational) remains the known live UB and has #561 open; itis not reached by anything this branch newly instruments or newly runs. If CI's
Qt/ladder legs surface anything this local run did not, it gets its own issue
with the real output, per
AGENTS.md— not a fix in this PR.No suppressions added, and no exemption widened. The only exemptions here
are ones that were already there and are now stated:
detailandqtin thespec-sync table,
morph_test_mainin the instrumentation check.Why one PR
All four are gate/CI-hygiene changes with no overlapping files except
ci.yml(#541/#542) andcheck_spec_citations.sh(#554). One CI cycle, onerebase, one merge.
Review notes (done inline, not by a forked reviewer)
apply_sanitizers()call site is guarded byif(DEFINED AF_SANITIZER), so the newFATAL_ERRORarm cannot fire on an ordinarybuild; verified across all 20 call sites.
before instrumenting them (
morph_qt_impl: ladder rungs,examples/common,examples/bank,tests/qt,tests/net_qt_interop,examples/qt_tls_client, and the WASM spike, which never definesAF_SANITIZER).linux-all-featuresbuildssoak/bench/forms_qmlonclang-debugandgcc-debug, whereAF_SANITIZERis undefined, so those blocks are inertthere.
yaml.safe_load);check_ci_clang_pin.shandcheck_rung_filters.shstill pass, and nothingin the tree referenced the removed
nmstep.docs/spec/, becausecheck_spec_citations.sh's dangling-reference scanreads
.shfiles too — caught by running that lint over this branch.🤖 Generated with Claude Code
https://claude.ai/code/session_01DbGrZGkr2WqcAs2DJyMym6