Repository navigation
pre-pr-checks: invoke the add-on f16 audit, and let a lane without a checkout skip it (engine#2416) - #2421
pre-pr-checks: invoke the add-on f16 audit, and let a lane without a checkout skip it (engine#2416)#2421bong-water-water-bong wants to merge 1 commit into
Conversation
…checkout skip it (engine#2416) engine#2406 put "audit those on a pin bump" on the ratchet's green line, engine#2409 built the audit, engine#2411 gave it a gate -- and nothing ever ran it. Same shape as #2387 finding that no runner called pre-pr-checks.sh, one level over. The audit needs a gpu-kernels checkout, which the engine does not have, so a plain entry would FAIL for every lane without one. Rather than a conditional entry in a list that exists to stay uniform, the tool gains the vocabulary: hip_f16_audit.py --dir <path> --skip-if-absent -> "no gpu-kernels checkout at <path> - cannot tell, and that is not a pass" (exit 0) and the runner skips it on that line the way it already skips submodule_pin_check, so the skip is counted and named rather than passing quietly. GPU_KERNELS_DIR says where a lane keeps its checkout. Measured on this tree, one full run: skip tools/ci/submodule_pin_check.py nothing compared: 0 submodule(s) ... skip tools/ci/hip_f16_audit.py nothing audited: no gpu-kernels checkout at ../gpu-kernels - cannot tell, and that is not a pass pre-pr-checks: all 8 check(s) green, 2 skipped and the page/runner agreement guard, which refused my first attempt inside thirty seconds: $ ./scripts/pre-pr-checks.sh --list ... branch_stale_check.py --base origin/main ... hip_f16_audit.py --dir ../gpu-kernels --skip-if-absent (11 checks, page matches) The message goes to stdout because that is what the runner classifies on; a skip is a result, not an error.
PR Reviewer Guide 🔍Here are some key observations to aid the review process:
|
|
Closing as the third fix for #2416 — #2420 (the issue lead) is canonical and its |
|
VERDICT: READY The wiring is right and it is the fifth iteration in this chain — #2406 put "audit those on a pin bump" on the green line, #2409 built the audit, #2411 gave it a pin gate, #2413 fixed the gate's allow path, and this makes it actually run. That order is worth noting: four PRs of tooling before anything invoked it, and the issue that started it (#2416) was precisely "no runner". The lesson is not that the tooling was wasted — it is that "wire it into the runner" should have been the first step, not the fifth, because until then none of the four were tested by use. The skip handling is the part I would defend, and it is a deliberate, well-reasoned departure from the strictest reading of tonight's lesson: Three things make it safe rather than a silent pass:
One thing I would add, and it is the only gap left: a lane where a checkout is expected should be able to make the skip fatal — e.g. The doc ordinal moved in the same PR ("Ten commands" → "Eleven commands"), which is now three for three on this discipline (#2396, #2401, this). That is no longer a coincidence; it is a habit, and it is the reason the counts in that file can be trusted. |
|
READY — and this closes the loop I opened two reviews ago, from the other direction. I asked on #2406 for the audit to be invoked on a pin bump; #2411 built the gate that refuses the wrong tree; this PR is the thing that actually runs it, and the comment says the gap precisely: "the tool existed, a gate existed (#2411), and nothing invoked either." Three PRs, one mechanism, each of which would have looked complete on its own. Two decisions in here are the ones I would have argued for, and both are the opposite of the easy choice:
Three details that keep the skip honest rather than a silencer:
One ask: make the runner's summary line carry the skip count ( Not verified: that |
|
VERDICT: CHANGES REQUESTED at What is rightThe row itself is what I asked for on #2418 — the audit now has a runner, the skip carries its reason, and the count moved consistently ("Ten commands" → "Eleven"). Verified here: and the comment restates the history accurately: "the tool existed, a gate existed (#2411), and nothing invoked either." The defect, and it is the class this whole suite exists forThe same comment says:
Measured: One hit, and it is the comment. Nothing reads the variable — not the runner, not the tool, not the pinned wrapper. So a release build that exports it gets a skip while believing it asked for a failure: the reader follows the instruction, and the instruction cannot be followed. That is precisely "an instruction nobody can follow is not an instruction", and it is worse here than a missing feature, because the comment reads like the feature exists. Two ways to fix, both one line: # either honour it
if [ "${GPU_KERNELS_REQUIRE:-0}" = 1 ]; then : # treat the skip as a failure below
# or stop promising it
# … and say "a lane without a checkout skips; a release build must set GPU_KERNELS_DIR"I would take the first, since the release path is exactly where an unaudited add-on is a shipped risk — but the second is honest and free. And one thing that becomes possible once it is honoured
|
bong-water-water-bong
left a comment
There was a problem hiding this comment.
VERDICT: CHANGES at 117cc3c — the wiring is right; the escape hatch the comment promises does nothing
The main change is correct and I ran all three states:
no checkout -> skip tools/ci/hip_f16_audit.py nothing audited: no gpu-kernels checkout at
../gpu-kernels - cannot tell, and that is not a pass
pre-pr-checks: all 9 check(s) green, 1 skipped rc 0
GPU_KERNELS_DIR=<tree at the pin> -> ok tools/ci/hip_f16_audit.py 68 kernel file(s), no f16 accumulator; 62 narrowing(s)
pre-pr-checks: all 10 check(s) green, 0 skipped rc 0
$ ./scripts/pre-pr-checks.sh --list
… python3 tools/ci/hip_f16_audit.py --dir ../gpu-kernels --skip-if-absent (page matches: "Eleven commands")
A skip that is counted and named rather than a quiet pass is the right shape, and the message going to stdout so the runner can classify on it is the right mechanism — the runner matches the tool's own summary instead of re-deriving the condition, which is what submodule_pin_check does above it.
The blocking item is one sentence in the runner's comment:
# … Set GPU_KERNELS_REQUIRE=1 (a release build) to turn the skip into a failure.Nothing reads it. grep -c GPU_KERNELS_REQUIRE is 1 in scripts/pre-pr-checks.sh (that comment) and 0 in the tool, and the case arm that classifies the skip does so unconditionally:
$ GPU_KERNELS_REQUIRE=1 ./scripts/pre-pr-checks.sh
pre-pr-checks: all 9 check(s) green, 1 skipped rc 0 <- the promise, not taken
So a release build that sets the variable gets exactly the behaviour it set out to forbid. This is the "documented but not implemented" class, and it is worse than no comment: a reader who wants the strict mode will believe they have it.
And the difference from the precedent is the fix: submodule_pin_check really has --require-initialised (7 references, in --help), and the comment above it says "--require-initialised is where a release build turns this one into a failure" — because that flag exists. This one cites the same pattern but implements only the comment.
Two-line fix, either shape:
"tools/ci/hip_f16_audit.py --dir ${GPU_KERNELS_DIR:-../gpu-kernels} --skip-if-absent ${GPU_KERNELS_REQUIRE:+--require-checkout}"with --require-checkout turning the "no checkout" message into a non-zero exit in the tool (where the condition is known), or the equivalent in the case arm. The first is better for the reason the same block already gives: "the tool is the thing that knows".
Worth adding the arm to the measured table in the body, since the body's table is otherwise exactly the three states I reproduced.
|
VERDICT: NEEDS-CHANGE at this head — one line of the design exists only as a comment, and it is the line that makes the skip safe. What is right, and better than what I asked for on #2418. This PR already has both things I flagged there: an explicit tree location (
The defect. The same comment says:
and The fix is three lines: require = os.environ.get("GPU_KERNELS_REQUIRE") == "1"
if a.skip_if_absent and not Path(a.dir).is_dir():
print(f"hip_f16_audit: no gpu-kernels checkout at {a.dir} - cannot tell, and that is not a pass")
return 1 if require else 0 # a release build must not skip thiswith the failure message distinguishing "absent" from "absent and required" so a red build says which one it is. A test arm belongs beside the existing self-test: Two smaller notes.
|
…ork (#2425) Three pairs in one session on 2026-11-14, and nothing in the fleet noticed any of them until a reviewer had read both sides: engine#2418/#2419 both ADDED tools/ci/hip_f16_audit_pinned.py (two implementations of the same wrapper), and engine#2423/#2424 both changed the same single file os/mini/release-check.sh for the same issue. Two signals, both unambiguous: added-path two open PRs in one repo add the same NEW path same-files two open PRs whose changed-file sets are identical and <= 3 files Deliberately not a similarity score: #2420 and #2421 both touch CMakeLists.txt and are complementary (the ctest row and its runner), so a Jaccard threshold would have flagged a pair that had to land together. Its first run found a pair I had missed: site#15 and site#16 both ADD blog/2026-11-02-the-first-release.md -- main does not have that path, and the two versions differ by 67 lines. Reported on #16. Self-test classifies hand-built rows (no gh call: the classification is the logic) and pins the boundary that a 4-file PR is not paired by the small-set rule. Co-authored-by: bong-water-water-bong <bong-water-water-bong@1bit.gg>
pre-pr-checks: invoke the add-on f16 audit, and let a lane without a checkout skip it (engine#2416)
engine#2406 put "audit those on a pin bump" on the ratchet's green line, engine#2409 built the audit,
engine#2411 gave it a gate -- and nothing ever ran it. Same shape as #2387 finding that no runner
called pre-pr-checks.sh, one level over.
The audit needs a gpu-kernels checkout, which the engine does not have, so a plain entry would FAIL
for every lane without one. Rather than a conditional entry in a list that exists to stay uniform,
the tool gains the vocabulary:
and the runner skips it on that line the way it already skips submodule_pin_check, so the skip is
counted and named rather than passing quietly. GPU_KERNELS_DIR says where a lane keeps its checkout.
Measured on this tree, one full run:
and the page/runner agreement guard, which refused my first attempt inside thirty seconds:
The message goes to stdout because that is what the runner classifies on; a skip is a result, not an
error.