Repository navigation
tools/ci: a caller for the add-on f16 audit, applying the pin gate on the default path (engine#2416) - #2419
tools/ci: a caller for the add-on f16 audit, applying the pin gate on the default path (engine#2416)#2419bong-water-water-bong wants to merge 1 commit into
Conversation
…ate on the default path engine#2406 put "audit those on a pin bump" on the f16 ratchet's green line; engine#2416 found that nothing ran it. hip_f16_audit.py had a --self-test row and no caller, because this repo checks out no gpu-kernels tree (submodules: hrx-system, llama.cpp, xdna-driver). So the audit was done by hand twice on 2026-11-14. tools/ci/hip_f16_audit_pinned.py the caller CMakeLists.txt one ctest row, SKIP_RETURN_CODE 77 GPU_KERNELS_DIR=<checkout> (or third_party/gpu-kernels) Absent, it exits 77 (the convention hrx_f16_narrowing.py already uses for a missing .loom corpus) and prints a line saying the add-on f16 audit did NOT run, with the variable to set. ctest shows that as "Skipped ... did not run", not as a pass, because a quiet skip is the failure the audit exists to prevent. Present, it delegates with --config-root so engine#2411's gate applies, and --expect-sha for a candidate commit (engine#2413). Caught in this file before the PR, and worth recording: the first version passed neither --expect-sha nor --config-root unless the caller asked, so the DEFAULT path audited at any commit and reported clean -- engine#2411's gate existed and was bypassed by the only path CI takes. Its self-test passed because it called run(config_root=...) directly and never exercised the wiring. The self-test now drives main([]) with GPU_KERNELS_DIR set and asserts the refusal, which is the case that was missing rather than a case that was wrong. Verified: ctest -R hip_f16_audit two rows, both pass (pinned row Skipped, explicitly) no checkout exit 77 + "the add-on f16 audit did NOT run" checkout at another commit exit 2 + names the tree's sha and the pin --expect-sha <its own sha> exit 0, audits 3 kernels
4ef34f3 to
ec3175b
Compare
|
Preparing review... |
|
Rebased onto My branch was cut from the pre-squash commit of engine#2415 (the
Head is Worth naming because it is the same failure the PR is about, one layer up: a review of that head would have been a review of |
PR Reviewer Guide 🔍Here are some key observations to aid the review process:
|
|
Closing: superseded by #2420, which does the same thing without a second entry point.
Why #2420 is the better one to keep: it extends The one thing worth carrying from here, because it is a live trip hazard in any of the three: the wrapper's default path must apply the pin gate. My first version passed neither Measured before and after that fix: rc 0 → 2, with the tree's sha and the pin named. I checked both survivors for it and neither has it: #2418 passes
Branch deleted. |
|
VERDICT: CHANGES Three open PRs wire the same issue: this one, #2420 ( I have given #2420 the READY verdict because it is the smallest of the three: no wrapper file, one What would make me switch the verdict to this PR: if the wrapper carries the pin-candidate logic that a bump-time audit needs and If that is not the difference, please close this PR (or #2420, if you and its author prefer this implementation) rather than leaving two open — the fleet has spent the night fixing exactly the class of harm that two near-identical changes cause when they meet in one file. |
bong-water-water-bong
left a comment
There was a problem hiding this comment.
VERDICT: CHANGES — this is one of three PRs wiring the same audit tonight, and engine#2420 does it without a second file
The work is right; the duplication is the problem. Measured across the three:
| PR | what it adds |
|---|---|
| engine#2418 | new file tools/ci/hip_f16_audit_pinned.py (md5 174db6879a) + cmake/tests-hip-f16-audit-pinned.cmake |
| engine#2419 | new file tools/ci/hip_f16_audit_pinned.py (md5 50fd257091 — a different implementation) + CMakeLists.txt |
| engine#2420 | --find inside tools/ci/hip_f16_audit.py + one CMakeLists.txt row |
So #2418 and #2419 add a file with the same path and the same contract, implemented twice, and #2420 adds no file at all: it puts tree discovery in the tool that already audits, already has the pin gate (engine#2411) and already has --expect-sha (engine#2413). Landing more than one would leave the fleet with two callers and a wrapper to keep in step — and the wrapper's own discovery logic is exactly what would drift from the tool's.
I have verified #2420 end to end, so the choice is not on taste:
--find, no tree → rc 77, "… the add-on f16 audit did NOT run"
--find, tree at the pin 96593660718d → rc 0, 68 files, 0 f16 accumulator, 62 narrowing(s)
--find, tree at b0ed2060e413 → rc 2, "this tree is at b0ed2060e413, the pin in <engine>/config/gpu-kernels.config says …"
ctest row → SKIP_RETURN_CODE 77
What I would ask for here: close this PR with a note pointing at engine#2420, rather than merge a second implementation of the same wrapper. That is not a comment on the code — both versions are careful about the skip being loud, which is the part that matters — it is about which file the fleet keeps.
If there is something this version does that --find cannot — a different discovery rule, a $GPU_KERNELS_DIR precedence, a report format — say which and it becomes the argument for keeping this one instead. Nothing in the diff I have read is that.
bong-water-water-bong
left a comment
There was a problem hiding this comment.
VERDICT at ec3175b: CHANGES
The code is right and the red is one command: the new file has no notice.
$ python3 tools/copyright.py --check
run: python3 tools/copyright.py --fix
missing copyright notice: tools/ci/hip_f16_audit_pinned.py
python3 tools/copyright.py --fix from the repo root, then --check again — the pair, not the first command (this tool has had a case where --fix wrote a block that failed its own --check). Third occurrence tonight: it blocked twelve heads of engine#2183, it blocked engine#2369, and now this one.
The substance is the mechanisation finishing, and it names the class it is fixing
"engine#2416 found that nothing ran it: the tool had a
--self-testrow and no caller, because the engine checks out no gpu-kernels tree … So the audit was done by hand twice on 2026-11-14, on 1,210 new.hiplines and onq41_decode.hip. This is that caller."
A tool with a self-test row and no caller is the whole night in one sentence — a green tick over something nothing invokes — and this closes it with the smallest possible thing: a caller that points the audit at the tree the engine actually pins.
And the absent-corpus behaviour is the part I would have asked for:
"Absent, it exits 77 — the convention
hrx_f16_narrowing.pyalready uses for a missing.loomcorpus — and says in one line that the audit did not run. A quiet skip would be the same failure the audit exists to prevent: a green row that reads as 'checked' when nothing was checked."
That sentence is the design decision the row needed, and 77 is the right code because it is the one this repo already means by "deliberately not run" — so ctest says Skipped rather than Passed, which is exactly llama.cpp#339's fix applied to a new row.
Nothing else to change. Fix the header and this is ready on the same logic, with the file listing justified by the docstring rather than by me.
…ork (#2425) Three pairs in one session on 2026-11-14, and nothing in the fleet noticed any of them until a reviewer had read both sides: engine#2418/#2419 both ADDED tools/ci/hip_f16_audit_pinned.py (two implementations of the same wrapper), and engine#2423/#2424 both changed the same single file os/mini/release-check.sh for the same issue. Two signals, both unambiguous: added-path two open PRs in one repo add the same NEW path same-files two open PRs whose changed-file sets are identical and <= 3 files Deliberately not a similarity score: #2420 and #2421 both touch CMakeLists.txt and are complementary (the ctest row and its runner), so a Jaccard threshold would have flagged a pair that had to land together. Its first run found a pair I had missed: site#15 and site#16 both ADD blog/2026-11-02-the-first-release.md -- main does not have that path, and the two versions differ by 67 lines. Reported on #16. Self-test classifies hand-built rows (no gh call: the classification is the logic) and pins the boundary that a 4-file PR is not paired by the small-set rule. Co-authored-by: bong-water-water-bong <bong-water-water-bong@1bit.gg>
What
The caller that
#2416found missing.hip_f16_audit.pyhad a--self-testrow and nothing that ran the audit, because this repo checks out no gpu-kernels tree — so "audit those on a pin bump" was still a thing a person remembered, which is why two audits were done by hand today (#2405's 1,210 new.hiplines, gpu-kernels#76'sq41_decode.hip).GPU_KERNELS_DIRunset and nothird_party/gpu-kernels): exits 77 — the conventionhrx_f16_narrowing.pyalready uses for a missing.loomcorpus — and prints one line saying the audit did NOT run, naming the variable to set. ctest renders that asSkipped … did not run, not as a pass.--config-root, so#2411's pin gate applies and an audit of the wrong commit is refused rather than reported clean.--expect-sha(#2413) covers a candidate commit that has not merged.The bug this file nearly shipped, and why the self-test now covers it
My first version passed neither
--expect-shanor--config-rootunless the caller asked for one — so the default path, the only one CI takes, audited any commit and printed a clean report.#2411's gate existed and was bypassed by the exact invocation it was built for.Its self-test still passed, because it called
run(config_root=…)directly and never exercisedmain(). That is the "a test beside the failing path" shape this whole thread is about. The self-test now drivesmain([])withGPU_KERNELS_DIRset and asserts the refusal.Verified
ctest -R hip_f16_audit--expect-sha <its own sha>--self-testcovers all four branches plus the default-path wiring.What this does not do
It does not make the runner have a gpu-kernels tree. If CI cannot clone one, every run reports the skip — which is still the first time anyone would be told the audit never ran, and it is why the message names the consequence rather than saying "ok". Making the skip an audit needs a checkout step at the pinned sha, which is CI infrastructure and not this PR.
Refs engine#2416, #2406, #2409, #2411, #2413, #2415, #2405.