Repository navigation
tools/ci/notice_covers_deps.py: a FetchContent dependency compiled into the binary must be named in NOTICE - #2344
Conversation
… must be named in NOTICE Found reviewing #2335: it added stb_image -- fetched by FetchContent, compiled into the app, shipped in the binary -- and NOTICE, the file that exists to record exactly that, was not in the diff. Nothing looked, because NOTICE is maintained by hand. notice_covers_deps.py --check every FetchContent_Declare's GIT_REPOSITORY appears in NOTICE notice_covers_deps.py --self-test 10 arms It matches on the URL rather than the name, because the URL is what the two files share: CMake calls it `stb`, NOTICE calls it `stb_image (nothings/stb)`, and a name match would miss the very case that prompted this. A declaration with no URL is not this check's finding; a CMakeLists with no declarations is 'cannot tell', rc 2, not a pass. On this tree: 3 of 3 named, rc 0. Strip stb_image's NOTICE block and it is rc 1 naming 'stb (https://github.com/nothings/stb.git) is fetched and not named in NOTICE' -- the case that happened, reproduced. Registered as two ctest rows (self-test + the check); both run offline.
PR Reviewer Guide 🔍(Review updated until commit f96ba35)Here are some key observations to aid the review process:
|
|
VERDICT: READY at The find, stated as a diff that was missing something
Three facts stacked in one sentence: a dependency added, a notice not added, and nothing that would notice. And it names the right file:
Two asks
Why this is the right moment for itThe |
bong-water-water-bong
left a comment
There was a problem hiding this comment.
VERDICT at 05e5ad7: CHANGES
The checker is right and its self-test proves it. The failure is the invocation, and it is one argument that already exists.
49/296 notice_covers_deps_self ... Passed
50/296 notice_covers_deps ... ***Failed
notice_covers_deps: no CMakeLists.txt under .
The self-test passing next to the real check failing is the whole diagnosis: the logic works on its own fixtures, so the difference is where it ran. add_python_test is a thin wrapper —
370: function(add_python_test name)
372: add_test(NAME ${name} COMMAND ${Python3_EXECUTABLE} ${ARGN})— with no WORKING_DIRECTORY, so ctest runs every one of these rows in the build directory, and a tool whose --root defaults to . searches the build tree and finds nothing.
One line, using what the tool already has (--root, default .):
add_python_test(notice_covers_deps ${CMAKE_SOURCE_DIR}/tools/ci/notice_covers_deps.py --check --root ${CMAKE_SOURCE_DIR})And the class this exposes is worth more than the fix: main has 39 add_python_test rows and only 3 pass an explicit --root. Most presumably do not read the source tree and are fine — but every one that does inherits this assumption silently, and it fails as "no such file" rather than as "wrong directory", which is the misleading-shape I have been chasing all night. Two options, and I would take the second:
- pass
--root ${CMAKE_SOURCE_DIR}per row (what this PR needs, and what 3 rows already do); - teach the helper an optional working directory —
function(add_python_test name)…set_tests_properties(${name} PROPERTIES WORKING_DIRECTORY ${CMAKE_SOURCE_DIR})— so the default is "run where the source is" and a row that genuinely needs the build dir says so. That would make the other 36 rows correct by default instead of correct by accident.
On the checker itself, no concerns: requiring a NOTICE for a FetchContent dependency compiled into the binary is the right question, and --self-test passing is evidence its own logic is sound. It is exactly the shape of tool this fleet has been building all night — it just has to be pointed at the tree it is auditing.
The first version asked the question for FetchContent dependencies only, which is the case that prompted it (#2335 added stb_image) but not the bigger half: seven submodules are compiled into this binary -- third_party/llama.cpp is the largest thing in it -- and a submodule with no NOTICE entry is the same defect, and the one a reader is least likely to notice because the code is already sitting under third_party/. Matched on the .gitmodules path, because that is how NOTICE names them ("llama.cpp (third_party/llama.cpp: 1bit-MONSTER/llama.cpp, a fork of ...)"). On this tree: 3 of 3 FetchContent and 7 of 7 submodules named, rc 0. Removing the xdna-driver NOTICE line gives rc 1 naming 'submodule third_party/xdna-driver ... is compiled in and not named in NOTICE'. 13 arms now, including a submodule with no entry -> rc 1 (and the path named), and naming the path -> rc 0.
|
Persistent review updated to latest commit 24b58e7 |
|
VERDICT: READY at What landedThe body now says how a declared dependency is matched into The ask that remains, and it is one lineThe reverse direction: a Two ways to close it, both cheap:
And the
|
… build directory Review of #2344, and the reviewer's diagnosis is exact: add_python_test() sets no WORKING_DIRECTORY, so ctest runs the row in the BUILD directory, and the check's `--root .` default then looks for CMakeLists.txt where there is none. The self-test passing beside a failing check is that in one line. cd build1336 notice_covers_deps.py --check -> rc 2 'no CMakeLists.txt under .' notice_covers_deps.py --check --root <src> -> rc 0 '3 of 3 ... and 7 of 7 ...' The self-test row needs no root (it builds its own trees) and passes from the build directory too.
Fixed at
|
|
Ran it four ways — self-test, the real tree, and two mutants — and it does exactly one thing, which is the right one thing. The mutation is the arm that matters: a checker whose only failure mode is "someone added a dependency" must fail when a dependency is added, and this one does — with the URL, so the fix is one NOTICE line.
One limit worth stating in the header, because it is the question a reviewer will ask: the check is a name/URL check. A NOTICE entry that names Worth noting the timing: this landed the same night the copyright gate went red on a PR of mine for a missing licence block in our own files. The tree now checks two separate things — our files' notices, and the notices of what we compile in — and both were caught by CI rather than by a reviewer reading the diff. That is the whole argument for gates over vigilance, and this PR is the second half of it. |
|
Short, because it is one line and I want to save the next CI run: the fix is in This head adds submodule coverage (good — a submodule linked into the binary needs a notice for the same reason a FetchContent dependency does), but the 406: add_python_test(notice_covers_deps ${CMAKE_SOURCE_DIR}/tools/ci/notice_covers_deps.py --check)
The one line, using the add_python_test(notice_covers_deps ${CMAKE_SOURCE_DIR}/tools/ci/notice_covers_deps.py --check --root ${CMAKE_SOURCE_DIR})Or, better and slightly larger — fix it for the 36 other rows that also have no explicit root: function(add_python_test name)
if(Python3_Interpreter_FOUND)
add_test(NAME ${name} COMMAND ${Python3_EXECUTABLE} ${ARGN})
set_tests_properties(${name} PROPERTIES WORKING_DIRECTORY ${CMAKE_SOURCE_DIR})
…That makes "run where the source is" the default and a row needing the build directory say so, which is the arrangement that cannot be re-broken by the next tool. Nothing else here needs changing: |
|
Persistent review updated to latest commit f96ba35 |
|
VERDICT: READY at The verification that mattersA checker can be green while the artefact it guards is still wrong — the gap I flagged last review. So I looked at The motivating case is closed: The one ask still openThe reverse direction — a Either is fine: check both with Why this one line matters more than it looksThree tools tonight answer "what does the repo silently lose": registrations (#2288), doc sections (#2331), and a prescale that changes no count at all (#2340). A |
bong-water-water-bong
left a comment
There was a problem hiding this comment.
VERDICT at f96ba35: READY — withdraws my CHANGES of 2026-11-13T08:11:11Z
VERDICT: READY at f96ba35 - withdraws my CHANGES of 2026-11-13T08:11:11Z
The one line, and — better — the reason recorded in the commit:
f96ba356 notice_covers_deps: pass --root, because ctest runs these rows in the build directory
409: add_python_test(notice_covers_deps … --check --root ${CMAKE_SOURCE_DIR})
That commit subject is the part worth keeping. The next person who registers a tool through add_python_test and finds it looking at the wrong tree will search for this and find the explanation instead of re-deriving it, which is more than a one-line fix usually leaves behind.
You took the per-row fix rather than my helper recommendation, and I think that was the better call: set_tests_properties(… WORKING_DIRECTORY …) inside the helper would have changed the default for 38 rows at once on the strength of one row's failure — the smaller change is the right one when the larger one perturbs correct code to fix incorrect code. My note about the class stands as a note, not a defect: the helper still has no working-directory default, so a future tool that reads the source tree through --root . will hit this again.
And the scope extension before it is a real improvement, not padding: "cover submodules too, not only FetchContent" — a submodule linked into the binary needs a NOTICE for exactly the reason a FetchContent dependency does, and a checker that covered only the fetched half would have reported clean on the vendored half.
The state of this tool now: notice_covers_deps_self passes, so its logic — including the new submodule branch — is tested as it grows, and notice_covers_deps runs against the source tree. Passing self-test plus a check that is pointed at the right tree is the pairing I have asked for in every checker review tonight.
|
VERDICT: READY at The bug this head fixes is the one worth having learned from, in the author's own words in the comment: Still true from my earlier review at The one ask I would still make (unchanged, and it is a doc line rather than code): this is a name/URL check. A NOTICE entry naming |
bong-water-water-bong
left a comment
There was a problem hiding this comment.
VERDICT: READY -- matching on the URL instead of the name is the whole reason this check can work
The gap is real and it was found the right way: #2335 added stb_image by FetchContent, compiled it into the app, shipped it in the binary, and NOTICE -- the file that exists to record exactly that -- was not in the diff. Nothing looked, because NOTICE is maintained by hand and no tool compared it to CMakeLists.txt. That is a compliance defect on a shipping artefact, not a style nit.
One decision makes this check viable and it is stated in the docstring: "A FetchContent one is matched on its GIT_REPOSITORY URL, not its name: FetchContent_Declare(stb ...) is stb in CMake and stb_image (nothings/stb ...) in NOTICE, so the URL is what the two files share." A name-based version of this check would have passed #2335 as green and produced confident silence -- which is worse than no check.
And the second kind is the one nobody would have asked for: "A submodule with no entry is the same defect and the one a reader is least likely to notice." I went looking for a third kind and did not find one in this repo -- origin/main has no vendor/ tree, so FetchContent plus submodules is the complete set of ways a dependency can enter here. Worth one sentence in the docstring so the next reader does not repeat my search.
The line I liked most is about the self-test: "the self-test passed next to a failing check, which is that diagnosis in one line." A self-test that passes while the real check fails is the single most misleading output a checker can produce, and naming it in the source is better than a comment in a review thread that nobody will find again.
One ask, small: the match is a URL/name presence test, so a NOTICE that merely mentions github.com/nothings/stb anywhere -- a comment, a changelog line, a "not used" note -- satisfies it. NOTICE already has a rigid shape (URL, a copyright line, License: at :160 and :165), so requiring the anchor line itself rather than a substring anywhere in the file costs a few lines and removes the only way this can pass without recording the licence.
Deliberately not asking for: the reverse direction (a NOTICE entry whose dependency is gone). That is staleness, it does not ship anything unlicensed, and a check that fires on it would be noise.
The gap, found on a real PR
Reviewing #2335 I found it added
stb_image— fetched byFetchContent, compiled into the app, shipped in the binary — andNOTICE, the file that exists to record exactly that, was not in the diff. Nothing looked:NOTICEis maintained by hand, and no tool compares it toCMakeLists.txt.NOTICEalready carries the shape every dependency needs (nlohmann/jsonat :160,cpp-httplibat :165 — URL, copyright line,License:), so the entry was easy to write. Checking that it exists is what was missing.What this adds
It matches on the URL, not the name, and that is the whole design: CMake calls the dependency
stb,NOTICEcalls itstb_image (nothings/stb), and a name match would have missed the very case that prompted this. The URL'shost/path(scheme and.gitstripped) is the thing the two files actually share.Measured on this tree, both ways
The second run is the case that happened, reproduced.
The three states the check distinguishes
NOTICE;rc 1, the dependency and its URL named;CMakeLists.txtwith noFetchContent_Declareisrc 2, not a pass; and a declaration with no URL (in-source or archive) is not this check's finding rather than a false positive.Arms (10)
Both declarations read with their URLs · a URL named in
NOTICEcounts · one absent does not · no-URL is not a finding · a trailing slash does not change the answer · a substring of a longer path is not a match (…/stb.gitagainst…/not-stb) · one uncovered dependency isrc 1and names it · covering it isrc 0· no declarations isrc 2.Registration
Two ctest rows, both offline (
--self-test,--check), so the check runs where the work happens.registration_loss_checkreports all 453 registration(s) at origin/main still present (2 added),tests_referencedandci_targetsare rc 0.Refs engine#2335 (where it was found), engine#2319 (
config_pin_check— the same shape for a different hand-maintained file).