Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
56 changes: 49 additions & 7 deletions tests/CMakeLists.txt
Original file line number Diff line number Diff line change
Expand Up @@ -216,13 +216,55 @@ include(Catch)
# test fails fast instead of stalling the whole CI job (observed: a single test
# hanging blocked a Windows runner for over half an hour).
#
# One test used to need an exception, registered separately by tag: the
# 70,000-node `equation()` depth test, whose O(n^2) string building cost 32.3s
# under ASan+UBSan and blew past 120s under TSan (#589, widened to 600s by
# #590). #582 made that rendering linear -- the same test now takes 0.11s under
# ASan+UBSan and 1.3s at -O0 under TSan -- so the exception, and the `[slow]`
# tag it keyed on, are gone again and every test is back under one cap.
catch_discover_tests(morph_tests DISCOVERY_MODE PRE_TEST PROPERTIES TIMEOUT 120)
# That number was never chosen with any particular test in mind, which is
# fine while every test is sub-second and wrong for the one that is not: the
# longest, most load-sensitive case in the binary was governed by the same
# ceiling as the cheapest (morph#760). So the blanket stays and `[slow]` is
# registered separately, exactly as it was for #589/#590 -- DISCOVERY_MODE
# PRE_TEST defers discovery to ctest invocation time, so a per-test TIMEOUT
# cannot be set with set_tests_properties() here (nothing is named that test
# yet at configure time); excluding a tag and giving it its own
# catch_discover_tests() call is the mechanism that is actually available.
#
# (#589's own use of `[slow]` is gone: the 70,000-node `equation()` depth test
# blew past 120s under TSan because of O(n^2) string building, #582 made that
# rendering linear, and it is back under the blanket cap. The tag is reused
# here for a different test and a different reason.)
catch_discover_tests(morph_tests DISCOVERY_MODE PRE_TEST TEST_SPEC "~[slow]" PROPERTIES TIMEOUT 120)

# `[slow]` (morph#760): `StrandExecutor never runs two tasks for one key
# concurrently under contention`. Not slow because it computes anything --
# 0.14 s on an idle box -- but because the strand serialises 3200 tasks per
# iteration and every handoff is a thread wakeup that has to wait its turn on
# the run queue. That cost is set by how contended the host is, and it climbs
# steeply. Measured on this tree, 12 cores, clang 22.1.8 Release, load made
# with plain spin loops, whole-case wall clock, one run each:
#
# run queue 1 (idle) -> 0.14 s
# run queue 14 -> 23.8 s
# run queue 27 -> 170.5 s
# run queue 38 -> 396.4 s <- load average 31-38 at measurement
#
# Every one of those runs *passed*: 40 assertions, `inFlight 1, maxInFlight 1`
# throughout. morph#760's report was this case being killed while passing, at
# load average 29-48 -- above the 38 measured here, and the curve is
# superlinear, so the fitted figure at a run queue of ~50 is roughly 730 s.
#
# 900 s is therefore ~2.3x the measured 396 s and ~1.2x that extrapolation. It
# is also the number this file already uses for its other load-sensitive case
# (`forms_schema_generation_is_not_route_count_sensitive`, below), chosen there
# for the same reason: a slow shared runner can be several times a workstation.
#
# What this deliberately does **not** do is reduce `kIterations`. Twenty
# iterations is this case's detection power for a rare interleaving; trading it
# away to fit a ceiling would make the case cheaper and worse at the only thing
# it exists for. See the comment on `kIterations` in test_strand_race.cpp.
#
# And what it does not buy is immunity: a host oversubscribed past roughly 4.5x
# will exceed 900 s too. The ceiling is here to catch a *deadlock* -- which is
# unbounded, not merely large -- and it still does, in 15 minutes instead of 2,
# for one test out of 3044.
catch_discover_tests(morph_tests DISCOVERY_MODE PRE_TEST TEST_SPEC "[slow]" PROPERTIES TIMEOUT 900)

# ── Two-binary journal-path skew test (issue #246) ───────────────────────────
# The executable form of the journal's data-at-rest contract, which
Expand Down
29 changes: 28 additions & 1 deletion tests/test_strand_race.cpp
Original file line number Diff line number Diff line change
Expand Up @@ -168,9 +168,36 @@ class DrainWatchdog {

} // namespace

TEST_CASE("StrandExecutor never runs two tasks for one key concurrently under contention", "[strand][race]") {
// `[slow]` (morph#760) is what gives this case its own ctest `TIMEOUT`; see
// tests/CMakeLists.txt, where the tag is excluded from the blanket 120 s and
// registered again with a budget sized from this case's measured loaded
// runtime. It is a *scheduling* budget, not a performance one -- see the note
// on `kIterations` below.
TEST_CASE("StrandExecutor never runs two tasks for one key concurrently under contention", "[strand][race][slow]") {
constexpr int kThreads = 8;
constexpr int kPostsPerThread = 400;
// Detection power, not a duration. Each iteration is one fresh
// pool/strand pair sampling the drain-and-re-arm interleaving once; twenty
// of them is how often this case gets to observe it. Cutting this number
// is the cheap way to fit a timeout and it makes the case worse at the one
// thing it exists for, so the budget was moved instead (morph#760).
//
// What the case actually costs is set by the *scheduler*, not by the work:
// the strand serialises `kThreads * kPostsPerThread` tasks, and each
// handoff is a wakeup that has to wait its turn on the run queue. Measured
// here, 12 cores, clang 22.1.8 Release, synthetic spin-loop load, whole
// case wall clock:
//
// run queue 1 (idle) -> 0.14 s
// run queue 14 -> 23.8 s
// run queue 27 -> 170.5 s
// run queue 38 -> 396.4 s
//
// Steeply superlinear in the oversubscription ratio, and the serialisation
// invariant held in every one of those runs -- `inFlight 1, maxInFlight 1`
// throughout, 40 assertions passed. A host busy enough will still exceed
// any fixed ceiling; that is a property of the measurement, not a defect
// this case can assert its way out of.
constexpr int kIterations = 20;

morph::exec::detail::ModelId const key{42};
Expand Down
Loading