No gate catches a ctest case sharing an on-disk database, and no CI leg runs ctest in parallel

[Yaraslaut]

The shape

Third data point in two days for the same failure: a remedy this repository
already worked out, recorded in prose, and then did not apply to the next
consumer.

• No gate catches a Q_OBJECT header split from its TU; the first signal is a link error in six CI legs #659 — the AUTOMOC header-pairing fix, recorded in cmake/morph_add_rung.cmake
  naming pastebin::app::App as where it was last hit. Recurred in
  examples/common/. It got a gate: scripts/check_qobject_moc_pairing.py.
• bank: the 21 bank_tests cases share one SQLite database and fail under ctest -j, with nothing in the tree saying so #682 — RESOURCE_LOCK for suites sharing an on-disk database, recorded in
  cmake/morph_add_rung.cmake:47-54 and applied at
  morph_add_rung.cmake:570 and examples/common/CMakeLists.txt:321. Bank did
  not apply it. Closed by #TBD, with per-process databases rather than a lock.
• In the same fix, examples/bank/tests/gui/test_bank_gui_qml_behaviour.cpp
  was found carrying a comment that asserted the wrong safety property —
  "wiped once per process rather than once per case, so a later case cannot
  delete the file an earlier one still has open" — when catch_discover_tests()
  makes the process be the case. A comment that states a false invariant is
  worse than none; it stopped the next reader looking.

Nothing in the tree can detect the next instance.

Two gaps, either of which would have caught #682

1. No check that a ctest case reaching a shared on-disk database is isolated.
A script could find the literals: a source file under a test target that names a
fixed path under temp_directory_path() (or an equivalent) and belongs to a
target registered with catch_discover_tests without a RESOURCE_LOCK. That is
the exact shape of #682 and of the ladder's own pre-existing hazard, and it is
the shape check_qobject_moc_pairing.py already proved is detectable.

2. CI never runs ctest in parallel, on any leg. This is the more interesting
one, and it is why #682 was latent rather than red. Measured on 7d4ca453
(clang-release, bank example configured):

$ ctest -j 12 -L bank
0% tests passed, 21 tests failed out of 21
HY000 (10) - [SQLite]disk I/O error (10)

$ ctest -L bank
100% tests passed out of 21

Three runs of three failed at -j 12; serial was green every time. Every CI leg
runs ctest serially, so the whole class of cross-case resource contention —
shared files, shared ports, shared temp paths — is invisible to CI by
construction, and visible on the first command a developer types. After the
fix, the same suite is 21/21 at -j 12 in 1.1s against 5.7s serial, so
parallelism is not only a correctness signal here but a real wall-clock saving.

Verification status

Reproduced, on 7d4ca453, Linux, clang 22.1.8, clang-release preset with
-DMORPH_BUILD_BANK_EXAMPLE=ON. The output above is real, not paraphrased.

Not verified:

• Whether any other suite in the tree fails under ctest -j. I ran -j 12
  only over -L bank. The ladder suites declare RESOURCE_LOCK morph_ladder_test_db and so are presumably fine, but I did not run them
  in parallel to confirm, and ladder: the ledger scenario corpus fails intermittently with "database is locked" — observed once, not reproduced #658 (SQLite database is locked in the ledger
  corpus) suggests the question is worth asking rather than assuming.
• Whether a parallel leg would be stable enough to gate on. A -j leg that
  flakes is worse than no leg, and CI runners have fewer cores than this box.
• The cost of either gate. I have not written or timed a detector script.

What would change the verdict

Close as invalid if a full ctest -j over the whole tree is already green and
expected to stay so — in which case the remaining question is only the missing
detector, and this should be narrowed to that.

Close as done when either a script rejects a test source that names a shared
database path without an isolation mechanism, or some CI leg runs its ctest
invocation with -j and would therefore have failed on 7d4ca453. The second
is the stronger test, because it needs no pattern to be guessed in advance.

Re-open if a fourth instance of "documented remedy, new consumer, not applied"
lands without a gate.

[Yaraslaut]

Triage: valid — both halves verified, and the second half is the structural one

No CI leg runs ctest in parallel. Measured across every invocation:

$ grep -rnE "ctest .*(-j|--parallel)" .github/workflows/*.yml
(no matches)

ctest invocations:  ci.yml 13,  drift-guard.yml 3

Sixteen ctest calls, none parallel. So the entire class of cross-case resource contention is invisible to CI by construction — which is the whole reason #682 sat latent until someone ran ctest -j 12 on a workstation.

triage: valid, bug, area: ci, area: ladder.

Why the missing gate matters more than the missing -j

#682 is fixed (per-case databases, landed in #686), but its fix has no regression guard. Nothing stops the next suite from hardcoding a shared path, and nothing would notice: CI is serial, so the symptom cannot appear there.

The third data point the #686 lane found is what makes this a gate rather than a note: test_bank_gui_qml_behaviour.cpp carried a comment asserting the wrong safety property — "wiped once per process rather than once per case" — which is exactly what fails when the process is the case. A comment claiming the invariant, while the invariant does not hold, is the shape this repository files issues about.

Two candidate fixes, and they are not equivalent

1. A gate: every ctest case reaching an on-disk database declares a lock, or uses a per-case path. Cheap and static, fits drift-guard.yml's "fast and dependency-free" contract — but it has to recognise "reaching a database", which is not syntactically obvious.
2. Run one leg with ctest -j: makes the whole class visible rather than this one instance, and needs no heuristic. Costs a leg, and would likely surface other latent contention immediately — the bank: a database per ctest case instead of one shared file, and a rounding that does not charge a whole minor unit for half of one (fixes #682, fixes #678, refs #685) #686 lane explicitly flagged that it only tested -L bank, and ladder: the ledger scenario corpus fails intermittently with "database is locked" — observed once, not reproduced #658 (SQLite database is locked in the ledger corpus) suggests there is more.

(2) is the one I would take, and the reason is that (1) can only catch what it knows to look for, while a parallel leg catches what nobody predicted. But it should be landed expecting a first bill, the way #646/#656/#679 were.

Do not do both without measuring — a gate plus a parallel leg is two mechanisms for one property, which is the state #674 is open about elsewhere.

Verification status

The -j absence and the 16 invocations are measured by me. The claim that a parallel leg would surface further contention is inferred from #658 and from the #686 lane's note that it tested only -L bank; nobody has run the full suite in parallel.

What would change the verdict

Close as wontfix if serial ctest is a deliberate invariant — it plausibly is, given RESOURCE_LOCK exists precisely to serialise within a parallel run, which only makes sense if someone once intended -j. Worth reading cmake/morph_add_rung.cmake:47's history before deciding; if the answer is "we chose serial", that belongs in a comment where the next person will find it.

🤖 Generated with Claude Code

https://claude.ai/code/session_01VptDWG2fKr2vBnLSJcgzgW