tests/.clang-tidy's suppressions also silence include/morph headers, so a header gate driven by test TUs under-reports

[Yaraslaut]

Summary

tests/.clang-tidy disables thirteen checks, each with a documented reason about
test idiom. But clang-tidy applies the configuration of the main translation
unit, not of the file a diagnostic lands in. So every one of those thirteen
suppressions also applies to include/morph/** headers, whenever the header is
reached from a TU under tests/.

That means a header finding's visibility depends on which TU happened to reach
it. The same finding in the same header is reported from an examples/ TU and
silently dropped from a tests/ TU.

This matters now for two reasons:

1. clang-tidy-diff's changed-lines scope hides 101 findings in include/morph/, each a trap for the next edit to that line #580's census gate proposal is directly affected. That census recommends
   gating on "all 60 [verify_interface_header_sets] stubs plus every TU under
   tests/". The tests/ half of that gate cannot report
   cppcoreguidelines-pro-bounds-avoid-unchecked-container-access — the check
   that is 250 of the census's 596 — in any header it reaches, nor
   bugprone-unchecked-optional-access (4 of the census's flagged possible
   defects), nor readability-identifier-length (58). A gate built to that
   recipe would be green on findings it was built to catch.
2. It is a second instance of this repository's named failure mode — a
   control that reports success while measuring nothing — and it is invisible
   from inside: tests/.clang-tidy is a correct and well-argued file about
   tests/, and nothing in it says it reaches further.

Verification status: reproduced

Revision 0e3b8823 (origin/master), clang-tidy 22.1.8, .clang-tidy
unmodified, compile database configured exactly as .github/workflows/ci.yml's
clang-tidy job configures it (ci.yml:2191, every optional feature ON).

The case is include/morph/forms/forms.hpp:496, which #627 records as leaking
two cppcoreguidelines-pro-bounds-avoid-unchecked-container-access findings past
a wrapped NOLINTNEXTLINE. It is inside forEachNamedMember, a template, so it
needs a TU that instantiates it.

From a tests/ TU that does instantiate it —
tests/test_views.cpp, whose own comments say it "exercises buildViewSchema's
forEachNamedMember walk":

$ clang-tidy -p build/clang-debug --quiet --extra-arg=-std=c++23 \
    --extra-arg=-Wno-missing-include-dirs tests/test_views.cpp

$ grep -c "unchecked-container-access" test_views.log
0

Zero, across the whole run — not just at :496. The check is off for the entire
include closure.

Other checks from the same TU are unaffected, so the run was real:

$ grep -E "forms\.hpp:(489|496)" test_views.log | sort -u
.../include/morph/core/../forms/forms.hpp:489:39: error: forwarding reference parameter 'action' is never forwarded inside the function body [cppcoreguidelines-missing-std-forward,-warnings-as-errors]
.../include/morph/core/../forms/forms.hpp:489:57: error: forwarding reference parameter 'visitor' is never forwarded inside the function body [cppcoreguidelines-missing-std-forward,-warnings-as-errors]

From a non-tests/ TU, examples/forms/gui_qml/FormsController.cpp, same
revision, same database, same command:

forms/forms.hpp:496:41: error: possibly unsafe 'operator[
forms/forms.hpp:496:66: error: possibly unsafe 'operator[

Both findings present.

tests/.clang-tidy carries InheritParentConfig: true and this in its
Checks: list, among twelve others:

  -bugprone-unchecked-optional-access,
  -cppcoreguidelines-pro-bounds-avoid-unchecked-container-access,
  -readability-identifier-length,
  -readability-function-cognitive-complexity,
  -readability-convert-member-functions-to-static,
  ...

Its own comment for that entry scopes the reasoning to test code — "these are
dom["key"] on the line after REQUIRE(dom.contains("key")) … the bound is
established by the assertion the check cannot read" — which is a correct
statement about tests/ and says nothing about include/morph/. The file's
header likewise says "Checks suppressed inside tests/ only", which is what
the author intended and not what clang-tidy does.

The same applies to the nine examples/*/tests/.clang-tidy copies, which carry
the same entries.

What I did not verify

• I did not measure how many of clang-tidy-diff's changed-lines scope hides 101 findings in include/morph/, each a trap for the next edit to that line #580's 596 findings this hides from a
  tests/-driven run. Establishing that means re-running the census twice —
  once over tests/ TUs only, once over the rest — and differencing. The census
  itself was a full-tree sweep over 695 TUs, so its 596 is not affected; only
  a gate built from a tests/-TU subset would be.
• I did not check whether clang-tidy has an option to apply the diagnostic
  file's config rather than the main file's. I believe it does not (the config
  is resolved once, from the TU path, before analysis), but I read no clang-tidy
  source.
• Not reproduced on the CI image, only locally on Arch Linux with clang-tidy
  22.1.8.
• This is not a claim that tests/.clang-tidy is wrong. Every entry in it is
  argued and, for test code, convincing. The defect is the unstated reach.

What would change the verdict

Close this when one of:

• the header gate contemplated by clang-tidy-diff's changed-lines scope hides 101 findings in include/morph/, each a trap for the next edit to that line #580 is built in a way that does not depend on
  a tests/-TU's configuration — e.g. gating only on the
  verify_interface_header_sets stubs, which sit under build/ and inherit the
  root .clang-tidy, plus non-tests/ user TUs for the instantiation-dependent
  residual; or
• tests/.clang-tidy's reach is narrowed so its suppressions cannot apply to
  include/morph/** (if clang-tidy offers a mechanism for that), or
• it is written down — in tests/.clang-tidy's own header and wherever the
  header gate is documented — that these thirteen checks are unreportable for
  any header reached from a test TU, with the list of which ones.

Re-open if a gate is later built on tests/ TUs without that record.

Related

• clang-tidy-diff's changed-lines scope hides 101 findings in include/morph/, each a trap for the next edit to that line #580 — the census whose gate recipe this affects.
• Four NOLINTNEXTLINE directives are wrapped onto two lines and suppress nothing; six findings leak past them #627 / lint: make wrapped NOLINT directives apply, gate the ones that cannot, and clear clang-tidy batch A (fixes #627, fixes #600) #631 — where this was found: forms.hpp:496's two findings could
  not be reproduced from the obvious tests/ TU, which is what surfaced it.
• clang-tidy: an error-severity finding sits in attributes.hpp that the changed-lines gate can never report #600 — the same shape one level up (a real finding the gate is structurally
  unable to report), for a different structural reason.

Found while verifying #627's fixes in PR #631. Filed rather than folded in, per
AGENTS.md: it is about the check configuration's scope, not about any of the
directives that PR repairs.

[Yaraslaut]

Triage: valid

Premise independently confirmed on master. tests/.clang-tidy disables the check by name, with its reason stated:

# cppcoreguidelines-pro-bounds-avoid-unchecked-container-access: same blind
# spot one level up -- these are `dom["key"]` on the line after
# REQUIRE(dom.contains("key")), and `wire[1]` after a REQUIRE on the frame's
# length. The bound is established by the assertion the check cannot read.

That reason is good — for test sources. The defect is that clang-tidy applies the main translation unit's config, so a suppression scoped by intent to tests/ also silences every include/morph/ header reached through a test TU. A decision about Catch2 idiom silently becomes a decision about production headers.

Scoped more narrowly than the issue states — and this matters

I checked which configs disable this check tree-wide:

$ for f in $(find . -name .clang-tidy -not -path './build*'); do
      grep -q "avoid-unchecked-container-access" $f && echo "SUPPRESSED in: $f"; done
SUPPRESSED in: ./tests/.clang-tidy

Only tests/. The examples/*/tests/, examples/bank/tests/ and examples/common/testkit/ copies that tests/.clang-tidy's header mentions carry the other entries (the Catch2 chained-comparison one), not this. So the blast radius is one directory, not the whole test estate.

What it does not undermine

#580's census of 596 stands. Its 695-TU run included the VERIFY_INTERFACE_HEADER_SETS stubs, which are generated into the build tree (build/<preset>/morph_verify_interface_header_sets/morph/*.hpp.cxx) and are therefore governed by the root .clang-tidy, not tests/'s. The examples TUs are likewise unaffected. So the 250 figure for this check was measured through configs that report it.

I am stating this explicitly because the issue's own framing — "a header gate driven by test TUs under-reports" — is right about the gate and could easily be misread as "the census is wrong." It is not.

What it does undermine, which is the real finding

#580's gate recipe. That report proposes the 60 header stubs (93%, 552 of 596) plus "adding TUs under tests/" to close the residual 44. That closure path cannot report this check — the one that is 250 of the census. So the gate as designed would be structurally blind to its largest category in exactly the region it was meant to extend into.

That is an AGENTS.md "verify rather than assert" case aimed at a gate that does not exist yet, which is the cheapest possible time to catch it.

Direction

Do not fix this by deleting the entry from tests/.clang-tidy — the stated reason is sound and the findings in test sources really are unreadable-bound idiom. Options, in the order I would try them:

1. Do not source header findings from tests/ TUs at all. The stubs already give 93%. Close the residual 44 with TUs that live somewhere governed by the root config — a small dedicated instantiation TU under tests/ would not qualify, but one under a new directory with no .clang-tidy, or a stub generated into the build tree, would.
2. Run the header gate with an explicit --config/--config-file, so the gate's check set is the root one regardless of which TU drives it. Verify it actually overrides rather than merges.
3. Narrow the suppression so it applies to test sources but not to headers they include — if clang-tidy offers no such scoping, say so, because that is the thing to write down.

Sequencing: this belongs with the gate work, which #580 correctly puts after the six remediation batches — a full-tree gate turned on while 500+ findings remain would keep master permanently red. Nothing here is urgent; it is a constraint on a design that has not been built.

What I did not verify

• I did not test whether --config-file overrides or merges with a directory .clang-tidy. Option 2 rests on that and it is unconfirmed.
• I did not re-run the filer's two measurements (forms.hpp:496 invisible from tests/test_views.cpp, visible from the examples TU). I verified the configuration that makes them necessary, not the observations themselves.
• I did not check whether any other check in tests/.clang-tidy also has a large header population — if one does, the same leak applies to it and the number would grow.

Labels: triage: valid, plus bug / area: ci already applied.

[Yaraslaut]

Sized, and one correction to the issue body

Worked on the branch laneCI-batch-621-632. Two things to record here: the
measurement this issue explicitly left open, and a factual error in the issue
about the examples/ copies.

The size of the under-report: measured

Verification status: reproduced and measured, on a8511aa6, clang-tidy
22.1.8, Arch Linux, against the compile database ci.yml's clang-tidy job
configures for itself (clang-debug, every optional feature ON — 703 entries,
134 distinct TUs under tests/). Each TU run twice, once with
tests/.clang-tidy in place and once with it moved aside, with the job's own
option set (--quiet --extra-arg=-std=c++23 --extra-arg=-Wno-missing-include-dirs -p build/clang-debug). All 134 TUs were
offered and analysed in both runs; no TU produced a clang-diagnostic-error
in either. Findings deduplicated on (file, line, column, check).

distinct findings inside include/morph/**, from those 134 TUs
  with tests/.clang-tidy in place ....  262
  with it removed ....................  595
  hidden by it .......................  333, across 25 headers

  247  cppcoreguidelines-pro-bounds-avoid-unchecked-container-access
   57  readability-identifier-length
    6  cppcoreguidelines-pro-bounds-array-to-pointer-decay
    5  modernize-avoid-c-arrays
    5  cppcoreguidelines-avoid-c-arrays
    5  bugprone-unchecked-optional-access
    3  cppcoreguidelines-pro-type-vararg
    3  readability-convert-member-functions-to-static
    2  readability-function-cognitive-complexity

  most affected: forms/forms.hpp 90, session/session_auth.hpp 61,
                 net/detail/sha1.hpp 32, forms/views.hpp 31,
                 util/quantity.hpp 19

The set difference is one-directional — 0 findings appear with the file present
that do not appear with it removed — which is what the mechanism predicts and
would not hold if the two runs differed for some other reason.

Four of the thirteen suppressed checks (bugprone-chained-comparison,
bugprone-misplaced-widening-cast, misc-use-internal-linkage,
readability-named-parameter) hid nothing in a header on this revision. That
is a property of today's headers, not of the reach, so all thirteen are
recorded.

The 595 figure bears on #580 and is posted there rather than acted on here.

Correction: the examples/ copies do not carry the same entries

The issue says "The same applies to the nine examples/*/tests/.clang-tidy
copies, which carry the same entries." They do not. All nine are byte-identical
to each other and each subtracts exactly one check:

$ md5sum examples/*/tests/.clang-tidy examples/common/testkit/.clang-tidy tests/.clang-tidy
205a4e7f64169d228d60ae5c20a751a7  examples/bank/tests/.clang-tidy
205a4e7f64169d228d60ae5c20a751a7  examples/bookmarks/tests/.clang-tidy
205a4e7f64169d228d60ae5c20a751a7  examples/crm/tests/.clang-tidy
205a4e7f64169d228d60ae5c20a751a7  examples/kanban/tests/.clang-tidy
205a4e7f64169d228d60ae5c20a751a7  examples/ledger/tests/.clang-tidy
205a4e7f64169d228d60ae5c20a751a7  examples/lims/tests/.clang-tidy
205a4e7f64169d228d60ae5c20a751a7  examples/pastebin/tests/.clang-tidy
205a4e7f64169d228d60ae5c20a751a7  examples/polls/tests/.clang-tidy
205a4e7f64169d228d60ae5c20a751a7  examples/common/testkit/.clang-tidy
f61ed8596557b99fe8835e04cb072ea1  tests/.clang-tidy

$ grep '^Checks:' examples/bank/tests/.clang-tidy
Checks: '-bugprone-chained-comparison'

Their reach is the same — the configuration is still resolved from the TU's
path, so that one check is off for headers too — but its effect is nil:
bugprone-chained-comparison fires only on Catch2's REQUIRE expansion, whose
diagnostic is attributed to the test source line, not to a morph header.

Measured over the 154 example test TUs in the same database, with and without
those nine files: 312 findings inside include/morph/** either way, zero
difference. That measurement on its own proves less than it looks:
bugprone-chained-comparison produced zero findings in either run,
because Catch2 3.16.0 locally carries the /* NOLINT(bugprone-chained- comparison) */ the config file's own comment describes, so the run cannot
separate "no header reach" from "the check never fired". The structural
argument is what carries the conclusion; the measurement only fails to
contradict it. Recorded in tests/.clang-tidy in exactly those terms.

What the fix does

Third close from the issue's list — the reach is written down, because it
cannot be narrowed (clang-tidy resolves configuration once, from the TU's
path, and offers no way to scope a suppression to the file a diagnostic lands
in).

• tests/.clang-tidy gains a section stating the reach, the measurement above,
  and a machine-readable header-reach: list naming all thirteen checks a
  test-TU-driven gate cannot report in a header.
• scripts/check_tidy_suppression_scope.sh gates both halves of that record:
  textually, that header-reach: names exactly what Checks: subtracts;
  behaviourally, that the reach still exists — a probe header under
  include/morph/ analysed from a TU under tests/ and from one that is not,
  using the repository's real .clang-tidy files, with the finding required
  absent from the first and present from the second. The second is the
  anti-vacuity control: the first assertion is a negative, which a probe that
  produces no findings at all satisfies by accident.
• Both run in the existing clang-tidy job, the only one with a pinned
  clang-tidy on it, before the Configure step — neither reads a compile
  database.

The behavioural probe reproduces the mechanism independently of the sweep: the
same two checks are reported against include/morph/probe_scope.hpp from
src/probe_scope_tu.cpp and not from tests/probe_scope_tu.cpp.

What is still not verified

• Not reproduced on the CI image; measured only locally on Arch Linux.
• I did not read clang-tidy's source. The mechanism is established
  behaviourally in both directions, not from the implementation.
• The gate does not measure the 333. It pins the mechanism and the record. A
  re-measurement is a 20-minute two-pass sweep and does not belong in CI.