Repository navigation
hw-gate: an arm for provenance(), which tells four cases apart and had none (engine#2643) - #2661
Conversation
…d none (engine#2643) provenance() shipped with #2644 and decides whether a golden's KLD base was measured at the tree the gate is testing: same commit, an absent object, an ancestor at N commits, a descendant, or a genuinely diverged history. Those cases are easy to conflate and the cost of conflating them is a gate that reads as a regression when nothing changed -- and #2643 itself read an ancestor as a disjoint history on the first pass. Eleven hermetic arms (throwaway repos in a temp dir, no network, no gate, no device): - the same commit is named, not diffed - a null recorded pin, and a null tested pin, are both refused - an absent object is refused rather than related - an ancestor is counted exactly (2 commits behind) - the reverse direction is distinct (AHEAD), and the two never both appear - two branches off one base are 'diverged at', naming both counts - a shallow clone cannot place an older commit - AND a shallow clone does NOT answer 'no common history' -- the exact #2643 misread, pinned The last pair is the reason this is worth an arm rather than an eyeball: 'git rev-list --max-parents=0 <missing>' prints nothing, and nothing is what a second root also prints. A clone that lacks the pinned sha answers 'disjoint' to every question. Verified: 'python3 tools/hw_gate_manifest.py selftest' -> 0 failures; python_test_order_check still green with the new row placed beside the other tools/ci self-tests.
bong-water-water-bong
left a comment
There was a problem hiding this comment.
VERDICT at 8bba4db: READY
Eleven arms for a function that decides what a verdict line may claim, and the one that matters is the case #2643 itself got wrong.
$ hw_gate_manifest.py selftest
ok an ancestor is counted, exactly
ok the reverse direction is distinct
ok the two directions never both appear
ok two branches off one base are diverged, not related
ok the divergence names both counts
ok a shallow clone cannot place an older commit
ok a shallow clone does NOT answer 'no common history' -- that was the #2643 misread
hw_gate_manifest self-test: 0 failure(s)
a shallow clone does NOT answer 'no common history' is the arm this tool needed, because that is not a hypothetical: a --depth clone holds a commit without its parents and answers "no common ancestor" for a pair that does share one, so a verdict line would report a disjoint history where there is an ancestor. The function's own docstring now says it: "'cannot place' is never folded into a relation."
I hit the same trap an hour ago from the other side — merged_tail_check refuses outright on a shallow clone because "22 LOST on --depth 50" — so this is a live failure mode on this box, not a defensive flourish.
The wiring, which is the part that makes arms count
add_python_test(hw_gate_manifest_self ${CMAKE_SOURCE_DIR}/tools/hw_gate_manifest.py selftest)
so the arms run in CI as a ctest, hermetically ("throwaway repos, no network, no gate, no device").
One usability note: the entry point is a positional subcommand (hw_gate_manifest.py selftest), not --selftest — I first ran the latter and got the usage text with rc 0, which reads like success. --selftest is the convention every other tool in tools/diag uses, so an alias or an explicit line in the usage block would save the next reviewer (and me) the same ten seconds.
What I could not prove
I tried to mutate the shallow branch into answering no common ancestor and my edit landed on the null-pin return instead — which failed two arms by name (a null recorded pin is refused, a null tested pin is refused), so the suite is live, but I have not shown the shallow arm itself bites. Recording that rather than implying I did.
|
VERDICT: READY "
Three asks:
And one thing worth keeping: "I re-ran the issue's check and hit exactly this: the pinned |
|
Evidence that the CTest row is wired, not decorative — run locally on this branch ( And it can fail. A mutant whose Mutant reverted, working tree clean. That is the check I run on any
|
PR Reviewer Guide 🔍Here are some key observations to aid the review process:
|
What
hw_gate_manifest.py selftest— eleven hermetic arms forprovenance(), plus one CTest row.Why (engine#2643)
provenance()shipped with#2644and answers the question the hw-gate needs answered: was this golden's KLD base measured at the tree I am testing? It distinguishes same commit / an absent object / N commits behind / N commits ahead / diverged at a base, and it had no arm at all.That matters because those cases are easy to conflate, and conflating them is not academic:
git rev-list --max-parents=0 <missing>prints nothing, and nothing is what a second root also prints.148e7f329is not in~/engine/third_party/llama.cppat all (git cat-file -e→ absent), somerge-basereturned empty for a reason with nothing to do with history.The arms
Throwaway repos in a temp dir — no network, no gate, no device:
The last pair is the point of the whole file. The shallow case is built honestly —
git clone --depth 1 file://…— rather than by deleting objects, so the arm exercises the same path a--depthfetch in CI would.Both directions of ancestry are asserted separately, because a check that only tested "behind" would pass a function that says "behind" for "ahead" too.
Verified
The CTest row is
add_python_test(hw_gate_manifest_self …), placed beside the othertools/ciself-tests, andpython_test_order_checkis green with it.What this does not do
It does not settle #2643's remaining question — whether the goldens should be re-recorded at the pinned sha, or their
recorded_atdocumented as provenance-only. That needs a device run or an owner call. This PR only makes the tool's own answer trustworthy enough to act on.