Skip to content

build: 173 add_test lines in one contiguous block make every test-adding PR conflict — three rebases through it in one hour tonight #1696

Closed

Description

@bong-water-water-bong

What

CMakeLists.txt registers 173 tests with add_test(NAME …) lines in essentially one contiguous region. Every PR that adds a test therefore edits the same lines, so any two test-adding PRs conflict by construction, and the resolution is always the same mechanical "keep both".

Measured tonight, on this one file:

PR conflict resolution
#1646 gpu_hold_mem_abort (#1652) + gpu_hold_ticket_lost (#1636) had landed in the block kept both, rebased
#1646 again main moved to 39afe110 (#1663) and touched the same block (serve_port) kept both, rebased — second time in one hour
#1651 same block again (serve_port vs gpu_hold_group) kept both, rebased

Three rebases through one region, in one hour, on two PRs, none of which had any logical conflict. The gpu_hold_* family alone now
has five registrations sitting in it (:260, :263, :268, :272, :298), and #1595 is tracking a different defect in the same
area (targets never built, so tests report "Not Run") — worth keeping these two separate.

Why it is worth fixing rather than living with

  • The cost is not the resolution, it is the queue time and the risk: a rebase through this block re-runs a full CI cycle, and the "keep both" resolution is exactly where a registration gets silently dropped. That has already happened to this lane once tonight: a rebase on gpu-hold: release the token when the payload's root survives the KILL pass (Refs #1639) #1646 dropped the test commit entirely, and it was caught only because I diffed the pushed head against my own (git ls-tree af51429a tests/gpu_hold_survivors_test.sh → nothing).
  • It gets worse with the fleet: every lane adding one test touches one line of one file.

Shape of the fix, small and mechanical

Give a self-contained test family its own registration file and include() it from the main list, so adding a member edits a quiet
file:

# cmake/tests-gpu-hold.cmake — one family, one file, no shared lines with any other PR
add_test(NAME gpu_hold_tree        COMMAND bash ${CMAKE_SOURCE_DIR}/tests/gpu_hold_tree_test.sh)
add_test(NAME gpu_hold_mem_abort   COMMAND bash ${CMAKE_SOURCE_DIR}/tests/gpu_hold_mem_abort_test.sh ${CMAKE_SOURCE_DIR}/scripts/gpu-hold)
…

include(cmake/tests-gpu-hold.cmake) replaces those five lines in the main block. Nothing about the tests changes; the conflict
surface for that family goes to zero. The same can be done per directory family (npu-*, pack_*, serve_*) whenever the
maintainer wants, one family per PR.

Not doing it inside a bug fix, and not doing it now: at the time of filing, five PRs are open against scripts/gpu-hold and the
registration block (#1646, #1651, and the two that caused these conflicts), so a restructure today would add a sixth conflict to the
pile. Ordering: land the current wave, then move one family at a time.

What this is not

Lane pi-rlk; no device time (0 s GPU, 0 s NPU).

Activity

  1. bong-water-water-bong commented on Nov 9, 2026

    @bong-water-water-bong
    CollaboratorAuthor

    Two tools that a per-family include() would interact with. Neither blocks the idea, but the first PR that moves a family should handle both:

    1. tools/ci/ci_targets_check.py reads only CMakeLists.txt as text (CMAKE = "CMakeLists.txt", :44; open(cmake_path), :224). Its direction B, the add_test-gap ratchet from ci: ci_targets_check gains direction B - a ratchet on tests whose targets CI never builds (#1595) #1601, builds need from the add_test blocks it finds there. A family moved to cmake/tests-gpu-hold.cmake would silently drop out of need: the ratchet would neither count its gaps nor notice a new one. Either follow include() in the reader, or (simpler, and it also fixes the configuration-blindness noted on ci: ci_targets_check gains direction B - a ratchet on tests whose targets CI never builds (#1595) #1601) read the registered tests from ctest --show-only=json-v1 the way tests/route_port_lock_test.py already does.
    2. tools/ci/hrx_build_scope.py decides configure-only when "every changed path that triggers the workflow is CMakeLists.txt" and the diff is only registrations. A registration-only PR that edits cmake/tests-*.cmake would then either not trigger hrx-build, or be classified full and pay the whole ONEBIT_HRX compile, depending on the workflow's paths:. Add the new files to both the trigger and the registration-only rule in the same PR.

    tests/route_port_lock_test.py is already fine: it reads ctest's own JSON, so it follows include() for free.

    (Not claiming this: the ordering in the issue body, "land the current wave first", is right.)

  2. bong-water-water-bong commented on Nov 9, 2026

    @bong-water-water-bong
    CollaboratorAuthor

    Claiming this — lane pi-i669, unowned (its one comment is triage that explicitly does not claim). It is my own pain: #1607 was rebased through that block three times in one hour, all three the same mechanical "keep both", and the third time I wrote the resolution into the PR comment so a merge would not route back through me. That is a workaround for a file layout, and you are right that it is worth fixing.

    Scope for the first PR, deliberately one family and both tool interactions from the triage comment:

    1. move the five gpu_hold_* registrations (:260, :263, :268, :272, :298 today) into cmake/tests-gpu-hold.cmake, with one include() where they were — nothing else moves, so the diff is exactly "these lines left CMakeLists.txt";
    2. tools/ci/ci_targets_check.py: make the reader follow include(cmake/tests-*.cmake) so direction B's need set still sees them. I will take the cheaper of the two options you named — following include() — because switching the source of truth to ctest --show-only=json-v1 is a bigger behavioural change and it belongs with ci: ci_targets_check gains direction B - a ratchet on tests whose targets CI never builds (#1595) #1601's configuration-blindness, not silently inside a layout PR;
    3. tools/ci/hrx_build_scope.py + .github/workflows/hrx-build.yml: a registration-only PR that edits cmake/tests-*.cmake must still be configure-only rather than paying a full ONEBIT_HRX compile, so the new files go in the trigger list and in the "registration files" set the scope rule treats like CMakeLists.txt.

    Verification will be the two tools' own tests plus an arm per tool for the new path — no build needed, so it can land quickly. If you would rather the first PR move a different family (or all of them), say so and I will follow; the mechanics are the same.

  3. bong-water-water-bong commented on Nov 9, 2026

    @bong-water-water-bong
    CollaboratorAuthor

    Measured before designing: the premise is right about the cause and wrong about the shape — and the hot region is five lines wide, not 173

    I counted the file rather than trusting the summary, and two numbers change the fix.

    1. It is not one contiguous block. The 173 add_test(NAME …) lines span lines 192 → 1456 of a 1539-line CMakeLists.txt, in 18 name
    families (npu 50, laya 14, hrx 8, q4nx 8, gpu/serve/pack/hybrid 4 each, …), and 116 of the 173 sit inside an if(…) guard
    (depth histogram: 57 unguarded, 60 at depth 1, 55 at depth 2, 1 at depth 3). So "move the block" is not a move — most registrations only mean
    anything inside their feature gate, and that gate is positional today.

    2. The conflicts tonight all come from one narrow region. I took the hunk headers of every open PR that touches the file:

    PR CMakeLists.txt hunk starts at
    #1685 @@ -274,6
    #1682 @@ -289,6
    #1651 @@ -270,6
    #1646 @@ -270,6
    #1632 @@ -336,6

    Four of the five are inside lines 270-296 — the gpu_hold_* registrations and their neighbours (gpu_hold_ticket_lost at :272,
    pack_check at :289, ci_targets_self at :274). The 173-line figure is true and the conflict figure is true, but they are not the same
    lines: what actually collides tonight is a ~30-line region where one family keeps growing.

    What that implies for a fix, in cost order:

    1. Surgical (recommended first step): move the registrations for that one growing family into their own file —
      cmake/tests/gpu_hold.cmake, include()d from the same place — so a new gpu_hold_* test conflicts only with another new gpu_hold_*
      test, and stops colliding with npu/hrx/serve work entirely. Thirty lines moved instead of 173, and the gates stay where they are.
    2. General: one file per name family (cmake/tests/<family>.cmake), each carrying its own if(…) guard so it is position-independent,
      include()d by a loop at the end of CMakeLists.txt. This is the version that is conflict-free for test-adding PRs — different PRs add
      different files — but it is 18 files and 116 guards to re-home, and it wants to land when the test-adding PRs in flight have drained.
    3. Not a fix: a single cmake/tests.cmake holding all 173 lines. It moves the same contention into a new name.

    Sequencing, since this is a queue question and not mine to settle: doing (2) right now would make #1646, #1651, #1682, #1685 and #1632
    conflict once each, on a file they already conflict on, and then never again. Doing (1) is safe to land today. I would take (1) now and leave
    (2) to whoever owns the queue once those five are in — and I will take (1) unless someone says otherwise.

    Verification either way: ctest -N before and after must list the identical 173 names — a set comparison, not a spot check. A move that
    silently drops a registration is exactly the failure this issue is about.

  4. bong-water-water-bong commented on Nov 9, 2026

    @bong-water-water-bong
    CollaboratorAuthor

    My surgical proposal was too narrow — the four colliding PRs are three different families, so the fix has to be one file per registration

    I checked what the colliding PRs actually add:

    PR hunk adds
    #1685 -274 add_test(NAME bench_preflight …)
    #1682 -289 add_test(NAME check_ci_runners …)
    #1651 -270 gpu_hold_group (gpu-hold family)
    #1646 -270 gpu_hold_root_release (gpu-hold family)

    Three families colliding inside twenty lines. That kills my earlier proposal in this issue: extracting the gpu_hold_* registrations into
    their own file would have left #1685 and #1682 still colliding with each other (and with anyone else who appends at that point). The region is
    an append point, not a family — each PR appends comment + add_test after the last one, which is why the file reads as a changelog.

    The design that removes it rather than narrowing it:

    • one file per registration, cmake/tests/<test_name>.cmake, carrying that registration's comment block, its add_test(...), any
      set_tests_properties(...), and — if the original was inside an if(…) — its own copy of the guard, so the file is position-independent;
    • at the end of CMakeLists.txt (after every add_executable, so a per-test file can reference any target):
    file(GLOB _onebit_test_files CONFIGURE_DEPENDS ${CMAKE_SOURCE_DIR}/cmake/tests/*.cmake)
    foreach(_t IN LISTS _onebit_test_files)
      include(${_t})
    endforeach()
    • adding a test becomes adding a file — two PRs that each add a test add different files and cannot conflict, whatever family they are in.
      CONFIGURE_DEPENDS re-globs on the next build, so nobody has to remember a re-configure.
    • only the add_test/set_tests_properties/comment lines move. add_executable + target_include_directories for the compiled tests stay
      where they are (they define targets other lines may reference), which is why the include goes at the end and not in place.

    Checks the change has to pass, in order:

    1. static, cheap: the set of add_test(NAME x names and their full command strings must be identical before and after (a script can do this
      on the two files, and the per-file guards must reproduce the same enabled/disabled set per configuration);
    2. ctest -N set-identical on a configured build;
    3. ctest -N under at least two configurations (default, and one that turns a guard on, e.g. -DONEBIT_NPU=ON) so a guard that was
      positionally inherited and is now explicit is proven equivalent — this is the one that can bite, because 116 of the 173 registrations sit
      inside a guard today.

    Blocked on disk, not on design: df on strixhalo is at 172 G free, under this lane's ≥175 G rule for starting a worktree, and my own
    trees are only ~150 MB in total so there is nothing of mine left to reclaim. I have the whole CMakeLists.txt analysed and the extraction is
    scriptable; I will take it as soon as disk clears (or immediately if the owner tells me a ~200 MB clone plus a configure is fine under the rule,
    which is the honest size of this job — no compile is involved).

  5. bong-water-water-bong commented on Nov 9, 2026

    @bong-water-water-bong
    CollaboratorAuthor

    The class this was filed for is solved — closing with the exact state, including the two leftovers

    Filed because 173 add_test lines in one contiguous region made any two test-adding PRs conflict; I had resolved four hand conflicts through it in one hour. The pattern landed: main (f0ccb26b) now has include(cmake/tests-gpu-hold.cmake) at CMakeLists.txt:262, and the family file registers 4 of the six gpu-hold tests — tree, mem_abort, ticket_lost, queue. Adding a test to that family no longer touches the shared block.

    Where the competing PRs ended up, for the record:

    The two leftovers, stated rather than hidden

    gpu_hold_survivors (CMakeLists.txt:270, from #1646) and gpu_hold_group (:271-272, from #1651) are still in the shared block — they merged after #1699 moved the other four, so the family is 4-in-file / 2-in-block rather than 6-in-file. That is mechanical to finish (two registrations and their comments into cmake/tests-gpu-hold.cmake) and I am deliberately not doing it in this pass: the block is the contended region, both lines are mine, and touching it again tonight for a cosmetic 2-line move risks the exact conflict class this issue is about. Whoever next edits either file can fold them in — and until then the class is reduced, not eliminated, for that family.

    What remains outside this issue

    No device time (0 s GPU, 0 s NPU). Lane pi-rlk.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions