Repository navigation
add_python_test: run the row in the source tree (37/37 measured before and after) - #2347
Conversation
…rows are about
Review of engine#2344, where the reviewer found the class: add_python_test() emits add_test() with no
WORKING_DIRECTORY, so every one of these 37 rows runs in the BUILD directory -- and a tool whose --root
defaults to "." then looks for CMakeLists.txt where there is none. The self-test passed beside the failing
check and the log said only 'no CMakeLists.txt under .', which is the diagnosis in one line.
Measured before and after, all 37 rows, from the build directory:
before (cwd = build): 37/37 passed, 19.70 s
with WORKING_DIRECTORY=source: 37/37 passed, 21.01 s
So no row depends on being in the build tree today, and the five rows that already pass an explicit
--root keep working (an absolute path is unaffected). A row that DOES need the build tree now has to say
so with its own WORKING_DIRECTORY, which is the right way round: the source tree is where the tools,
tests/ and config/ these rows read actually live.
PR Reviewer Guide 🔍(Review updated until commit 671c5bc)Here are some key observations to aid the review process:
|
|
VERDICT: READY at The mechanism, stated so it cannot be forgotten
That is a harness-wide defect, not a tool bug: every python test row ever added ran somewhere other than the source tree, so any tool that defaults And the run is quantified: 37/37 measured — so the fix's evidence is that all rows now run in the source tree and still pass, which is the correct shape (a change to where things run must not change whether they pass). Two asks
Why a five-line change is worth a verdictBecause it is the difference between "the tool ran" and "the tool ran somewhere" — and every future python-tool check inherits whichever one this PR establishes. A harness default is the highest-leverage five lines in the tree. |
bong-water-water-bong
left a comment
There was a problem hiding this comment.
VERDICT at 2cc1c2d: READY
The measurement is the whole PR, and it is the measurement I said this change would need:
| result | |
|---|---|
| before (cwd = build) | 37/37 passed, 19.70 s |
with WORKING_DIRECTORY = ${CMAKE_SOURCE_DIR} |
37/37 passed, 21.01 s |
37 of 37 both ways, with timings — which settles the one risk I named in the #2344 review: a tool that writes into the build tree would break when moved. It is measured absent rather than assumed absent, for every row, and the +1.3 s total is the cost of the change and is stated.
And you checked the neighbourhood too, which is the part that makes a 42-row change safe to believe: registration_loss_check reports "all 470 registration(s) at origin/main still present" and copyright --check rc 0 — so the ratchets that could have been disturbed by a context change were checked rather than trusted.
Splitting it from #2344 was the right call and I want to say why clearly: the smaller fix shipped in minutes and unblocked a red build; the default change ships with evidence, as a decision someone can disagree with on its merits. That is what the two options should have looked like from the start — I offered them as alternatives in that review, and doing both in sequence was better than either.
The end state is the one that cannot be re-broken: a new tool registered through add_python_test now runs in the source tree by default, and a row that genuinely needs the build directory says so. The five rows that already pass an explicit --root are unaffected, and the 37 that did not are now correct by default instead of correct by accident.
One small thing worth keeping in the helper's comment: the reason (ctest runs rows in the build directory, so a tool reading the source tree must be told where it is) — because in six months the set_tests_properties line looks redundant to anyone who has not been bitten by it, and deleting it would restore the bug silently.
|
VERDICT: READY — it generalises the fix I got wrong in #2344, and I ran the rows it touches in a fresh configure. Setting The comment earns its place twice: it names the symptom that took a reviewer to diagnose tonight ("the self-test passed beside a failing check and the log said only Nothing in the tree currently overrides back: the 11 other |
|
Persistent review updated to latest commit 671c5bc |
|
VERDICT: READY at Standing: If the "before" behaviour is now stated (what the rows were doing and whether any passed while reading an empty tree) and One line still worth having if it is not there: which rows, if any, were passing while reading nothing — the |
|
The class, from a review of #2344
add_python_test()emitsadd_test()with noWORKING_DIRECTORY, so every one of its rows runs in the build directory. A tool whose--rootdefaults to"."then looks forCMakeLists.txtwhere there is none:The self-test passing beside the failing check is the whole diagnosis — the logic works on its own fixtures, so the difference is the invocation.
The change, and the measurement that makes it safe
All 37 rows, from the build directory, before and after:
WORKING_DIRECTORY = ${CMAKE_SOURCE_DIR}So no row depends on being in the build tree today, and the five rows that already pass an explicit
--rootare unaffected (an absolute path does not care about cwd). A row that does need the build tree now has to say so with its ownWORKING_DIRECTORY— which is the right way round:tools/,tests/andconfig/, the things these rows read, live in the source tree.registration_loss_check: all 470 registration(s) atorigin/mainstill present.copyright --checkrc 0.Why a separate PR rather than the one that found it
I said in the #2344 thread that teaching the helper a working directory changes 42 rows' runtime context at once, and that it belonged in its own PR with the rows inspected. This is that PR, and the inspection is the table above: measured, not reasoned.
Refs engine#2344 (where it was found), engine#2319/#2344 (
--root-taking tools), engine#2313 (pre-pr-checks.sh, which runs five of these checks by hand and can therefore be the place a lane notices).