Skip to content

test(gate): attribute an installed-vs-source parity failure to the machine, not the repo (BACKLOG #1367) - #632

Merged
wshallwshall merged 2 commits into
mainfrom
claude/builder-1-1367-gate-parity
Aug 27, 2026
Merged

test(gate): attribute an installed-vs-source parity failure to the machine, not the repo (BACKLOG #1367)#632
wshallwshall merged 2 commits into
mainfrom
claude/builder-1-1367-gate-parity

Conversation

@wshallwshall

Copy link
Copy Markdown
Collaborator

Built by Builder 1. Landed by the Lander. Two files, +257/-0: docs/BACKLOG.md and tests/test_gate_installed_parity.py.

Handed over with two full-suite failures on the record, deliberately

The author asked that these be visible rather than buried, and they are right that it matters:

run 1   1 failed  13851 passed   tests/test_ledger_check.py::test_ci_mode_works_on_a_SHALLOW_clone...
run 2   1 failed  13851 passed   tests/test_selfheal_installed_parity.py::test_the_installed_selfheal_payload...

Two consecutive full runs of the same tree, two different tests, neither touched by this branch. A defect in the change would reproduce. These do not, and the non-reproduction across different tests is the evidence -- neither verdict alone would be.

What they established about each, so nobody re-derives it:

Run 1 passes in isolation on this branch, first try. The cause is unnameable from the log: the test runs git clone --depth=1 with capture_output=True, check=True and no handler, so git's stderr lands inside a CalledProcessError whose repr omits it. Exit 128, no message, at exactly the moment the message is the diagnosis. That is a real test defect independent of this run, and it is unallocated.

Run 2 is the installed-vs-source parity family the Cleaner root-caused. The author applied the Cleaner's corrected predicate rather than the retired commit-count one: git diff --quiet origin/main -- <source> reports this checkout's worktree-selfheal.ps1 matches origin/main exactly. So the checkout is not the odd one out -- the installed payload on that machine is. Machine state, not repository state, and precisely the case where reinstalling is the wrong move.

They did not run a third time, and say why: two runs at 47 minutes each already cost more than the finding, and a third could not separate the hypotheses either, since both are environmental and neither is in the diff. I accept that. CI is the arbiter here and it runs on a clean host.

An attribution trap worth more than the change

The author nearly reported this branch as touching three files, including .github/workflows/codeql.yml, which they never went near. It appeared because they were 1 commit behind and a two-dot diff renders main's own new commit as a difference.

git diff --name-only origin/main..HEAD                          3 files  <- wrong
git diff --name-only $(git merge-base origin/main HEAD)..HEAD   2 files  <- right

A two-dot diff against a moving branch conflates "what I changed" with "what main changed", and it reads as scope creep in your own PR. I reproduced both forms; they agree at 2 files now only because the author merged main in first.

Lander verification

ahead of origin/main    3        behind    0
merge-tree vs main      exit 0 (clean)
diff                    2 files changed, 257 insertions(+)
'+## 1367' in ledger    1

Before the push, git branch -r --contains 38f8509d9 named only the private rescue ref. Fourth branch tonight where a count-based check would have read "already pushed."

Not armed

Required approvals are 0, and this touches gate test infrastructure.

Generated with Claude Code

wshallwshall and others added 2 commits August 27, 2026 06:37
…t (BACKLOG #1367)

The message was already right, and that is the point. It refuses to name a
culprit, correctly, and tells the reader to work out which copy is older first,
warning that installing from an older checkout downgrades a machine-global file.
Three seats skipped that step in one day. One reinstalled a gate that was already
correct and left 121 worktrees ungoverned for some minutes.

An instruction that must be followed under time pressure is one that will not be.
So the attribution is computed here instead of requested.

STRICTLY ADDITIVE. What is compared does not change, the assertion does not move.
A verdict is added to the failure text from origin/main's blob of the same path:

  installed == main, checkout != main   YOUR CHECKOUT is the odd one out --
                                        behind, or legitimately changing the
                                        gate. Do not reinstall.
  checkout == main, installed != main   THE INSTALLED COPY is the odd one out.
                                        What the test exists for.
  neither == main                       Unattributable, the strongest reading.
  main unreadable                       ATTRIBUTION UNAVAILABLE, said plainly.

THE TWO REJECTED SHAPES, because rejecting them is most of the work.

Comparing against origin/main INSTEAD would go green when the installed copy is
TAMPERED WITH, so long as it matches main -- strictly worse than the problem
being solved -- and would red a PR legitimately changing the gate.

Skipping when the checkout is behind risks a FALSE GREEN, and this file's own
negative control exists to prevent exactly that: its docstring says folding CRLF
out of the comparison was "precisely the edit that could turn a false RED into a
false GREEN". Spending a guard to quiet a message is the trade this project
refuses everywhere else.

ADDITIVE ATTRIBUTION CANNOT PRODUCE A FALSE GREEN, BECAUSE THE ASSERTION IS
UNTOUCHED. That is the whole argument.

TWO PREMISES IN THE DISPATCH WERE WRONG AND ARE MEASURED IN THE ROW. Distance
from main is not the predicate -- 11 behind, 7 passed, 0 gate files changed in
those commits. The gate SOURCE has to have moved. And "2 failed, 22 passed" is
24 tests where this file has 7, so it describes a wider selection.

The unreadable branch is the one most easily dropped for tidiness and has its own
test saying why it must not be: a computed-looking verdict is trusted more than a
request to go and check. A fifth test asserts the four branches are mutually
distinct, since four branches returning one string would pass every other
assertion in the block. A sixth drives the REAL assertion through a temporary
installed path to prove the message actually calls the verdict -- the machine-
global file is never read or written for it.

Mutation-tested: dropping the verdict from the message reds only the wiring test;
collapsing the unreadable branch reds only its own.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
…es it (BACKLOG #1367)

The attribution said WHICH SIDE drifted. It did not say which FILE, or how to
check -- and the obvious instrument does not answer the question.

DISTANCE IS NOT THE PREDICATE. Measured: an 11-commits-behind tree passes all
seven of these, because the gate source did not move in those eleven. A tree 40
behind whose gate files are untouched is green; a tree 1 behind whose gate file
moved is red. So a reader reaching for

  git rev-list --count HEAD..origin/main

gets a number that decides nothing. A fleet note recorded "31 commits behind" as
though the number were the cause; it was the circumstance that made it true once.

The checkout-is-odd branch now prints the file and the control that does decide:

  git diff --quiet origin/main -- <path>    # exit 0 = your tree matches main

Suggested by the Cleaner, whose own episode is why the attribution exists, and
whose note carried the commit-count framing this replaces. It is worth more than
when it was built: since the cause is not distance, a reader cannot derive it
from a commit count at all, so the test has to say it.

Asserted, with a negative arm: the verdict must NOT offer rev-list, because that
is the instrument that does not answer this. Mutation-tested -- deleting the
control line reds only that test.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
@wshallwshall
wshallwshall force-pushed the claude/builder-1-1367-gate-parity branch from 5ffde1e to 3fc2d2e Compare August 27, 2026 11:39
@wshallwshall
wshallwshall added this pull request to the merge queue Aug 27, 2026
Merged via the queue into main with commit 58e710a Aug 27, 2026
39 of 40 checks passed
@wshallwshall
wshallwshall deleted the claude/builder-1-1367-gate-parity branch August 27, 2026 12:08
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant