Nothing checks that a gate left the worktree unmodified, and Exit-0 → pass would report a repairing gate as a clean run - #151
Merged
Merged
Conversation
…→ pass` would report a repairing gate as a clean run Closes #41
…d `Exit-0 → pass` would report a repairing gate as a clean run
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to subscribe to this conversation on GitHub.
Already have an account?
Sign in.
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Plan
Plan: Worktree modification check around gate execution (#41)
Normative check
Consistent.
docs/gates-and-commands.md§ "The non-modification rule is checked, not assumed" (line 160) requires exactly this:OutcomeBrokeContractis declared atgate.go:52. The outcome table (line 118) assigns this cause: "modified the subject it measured." No document contradicts. No document update needed — the rule is already stated; this implements it.Change 1 —
pkg/backend/github/worktree.go: wrapRunGatewith before/after comparisonCurrent method (lines 182–188) validates the name, builds argv, and delegates to
runGate. The new version:Before the gate: call
w.b.git.StatusPorcelain(ctx)and store the result asbefore. If this fails, return an error (no gate ran, no outcome exists — the runner cannot fulfil its contract without a starting snapshot).Run the gate: call
runGateexactly as today.Short-circuit on error or non-measured outcome: if
runGatereturned an error, or the outcome is anything other thanOutcomeMeasured, return immediately. The four non-measured outcomes are already failures; overriding them would lose attribution (e.g., timeout → broke-contract sends the wrong person to investigate).After the gate: call
w.b.git.StatusPorcelain(ctx)again, store asafter. If this fails, setrun.Outcome = OutcomeBrokeContractwith a Detail naming the git error — the gate did run and claimed to measure, but the runner cannot verify integrity, so the safe direction is refusal.Compare: if
before != after, setrun.Outcome = OutcomeBrokeContractandrun.Detail = "the gate modified the worktree:\n" + after.Return
run.Why
StatusPorcelainand notIsDirtyIsDirty(git.go:111) uses--untracked-files=no— it misses new files a gate drops into the tree. The issue and normative doc both require untracked non-ignored files to count.StatusPorcelain(git.go:119) uses--untracked-files=normal, which is the correct boundary. Git-ignored files are excluded by git itself, matching the doc: "paths the project ignores are outside the subject."Why only override
measuredA timed-out gate that also dirtied the tree is still a timeout for retry purposes. A died gate is still died for attribution. The worktree is still "spent" in both cases, but that enforcement (refusing subsequent gates) is out of scope — see below.
Change 2 —
pkg/backend/github/gate_test.go: updategateWorktreedummy runnerThe existing dummy runner (line 49) calls
t.Errorffor ANY command through the runner, which will now fire forStatusPorcelain's git calls. Update it to allowgit statuswhile still catching unintended calls:This returns empty output (= clean tree) for status calls, so
before == after == ""and all existing outcome tests pass unchanged. The guard still catches any other git command being called from the gate-spawning path.Change 3 —
pkg/backend/github/gate_test.go: new test helper and test casesNew helper:
gateWorktreeGit(t, script, timeout) (*worktree, string)Like
gateWorktree, but initializes a real git repo:git init,git add -A,git commit -m "init"so the tree starts clean.bin/gatewith the script.beforesnapshot, but the gate doesn't modify it, sobefore == afterstill holds). Actually — the script must be committed or elsebeforewill list it as untracked,afterwill too (unchanged), andbefore == after. Either way works, but committing is cleaner.Backendwith a realgitOps(vianewGitOps(dir)).Test cases
All in a single
TestRunGate_DetectsWorktreeModificationtable-driven test:echo '{"g":1}'OutcomeMeasuredecho x >> tracked; echo '{"g":1}'OutcomeBrokeContracttouch stray; echo '{"g":1}'OutcomeBrokeContractecho x >> tracked; git checkout -- tracked; echo '{"g":1}'OutcomeMeasuredtouch stray; exec sleep 60OutcomeTimedOuttouch stray; exit 0OutcomeDiedThe helper creates a committed file named (e.g.)
trackedso the "modifies tracked file" and "modifies and restores" cases have something to work with.What this does NOT do
Spent-worktree enforcement. The normative doc says remaining gates must not run in a modified worktree. There is no multi-gate loop today —
runIntegrationGateruns one gate, and any non-measured outcome already abandons the resolution. Enforcement belongs in the loop that doesn't yet exist, and pre-building it would be speculative structure.No change to
runGate(). It is a pure process runner with no git knowledge. The modification check is the worktree layer's concern.No change to the
flow.Worktreeinterface orGateRunstruct. The existingOutcomeBrokeContractandDetailfield are sufficient.No change to the fake backend (
pkg/backend/fake/). It doesn't spawn processes and has no worktree to check.No change to downstream consumers (
runIntegrationGate,CheckFit). Both already handleOutcomeBrokeContractcorrectly — it's a non-measured outcome, and all non-measured outcomes are treated as gate failure.Verification
go test ./pkg/backend/github/ -run TestRunGate— all existing tests pass with the dummy runner update; new modification-detection tests pass.go test ./...— full suite, no regressions.bin/verify— the project's own gate, which is what the flow runs before opening a PR.Review
bin/verifypasses clean.Review
Correctness
The implementation does what the issue asked:
StatusPorcelain(which runsgit status --porcelain --untracked-files=normal) atworktree.go:190.worktree.go:208-218.OutcomeBrokeContracton difference, neverOutcomeMeasured—worktree.go:215-218.OutcomeMeasured—worktree.go:203-206. Non-measured outcomes (timed out, died, could not start, broke contract from bad envelope) are already failures and keep their attribution. This is correct: overridingtimed_outwithbroke_contractwould send the wrong person to investigate.Untracked, non-ignored files count as a change — confirmed:
StatusPorcelainuses--untracked-files=normal, which reports??entries for untracked non-ignored files. The test at line 458-461 ("creates untracked file") verifies this."Modified and restored passes" — confirmed by the test at line 463-466, which matches the document's stated limit (line 172).
The mock in
gateWorktree(the non-git helper) is updated correctly: it interceptsstatusargs to return clean, so existing tests that don't use a real git repo continue to work without the new check interfering.Edge path I checked: if
StatusPorcelainfails after the gate (e.g.,.gitcorrupted by the gate), the implementation returnsOutcomeBrokeContractwith a diagnostic rather than propagating an error — correct, since a gate did run and did produce something.Scope
The diff touches exactly two files, both in
pkg/backend/github/. No unrelated changes.Duplication
StatusPorcelainalready existed ingit.go:122and is reused here. No new source of truth.Tests
Six test cases covering:
measured(positive control)broke_contractbroke_contractmeasured(the stated limit)timed_out(attribution preserved)died(attribution preserved)Each would fail if the check were reverted: the modify/untracked cases would return
measuredinstead ofbroke_contract. ThegateWorktreeGithelper correctly initializes a real git repo with a tracked file and committed state soStatusPorcelainproduces meaningful output.Workarounds
None. The implementation is direct.
Against the normative document
docs/gates-and-commands.mdlines 160-172: the implementation matches. "The subject is the tracked tree" —StatusPorcelainwith--untracked-files=normalcaptures exactly this (tracked changes + untracked non-ignored; ignored paths excluded). "A difference means broke the contract, never measured" — enforced at line 215-218. The two stated limits (modify-and-restore passes; detection is after the fact) are both accepted by the implementation.What I deliberately left
Step 4 of the issue — "Refuse the remaining gates for that transition in that worktree" — is not implemented in
RunGateand does not need to be:RunGateis the per-gate boundary. The caller (runIntegrationGateinissue/steps.go:721-743) already treats any non-measured outcome as an error that stops the step, so no further gates run in the same worktree after abroke_contract. When integration becomes a composition of multiple gates, the loop that iterates them will need to check for this — but that composition does not exist yet, so there is nothing to wire.Nothing to fix. The change is correct, scoped, tested, and aligned with the normative document.
Coverage
bin/verifypasses cleanly.Already covered by the existing
TestRunGate_DetectsWorktreeModification(shipped with the implementation):broke_contractbroke_contractAdded:
TestRunGate_DetectsTrackedFileDeletion— a deletion is a modification. Without this, someone could remove thebefore != aftercomparison for deletions specifically (e.g. by comparing only added lines) and no test would fail.TestRunGate_ModificationDetailNamesTheChangedFiles— asserts that theDetailfield carries the porcelain output naming each changed file. Without this, the detail could be empty or generic and the outcome would still pass the existing table-driven test, leaving a human with no way to know what was changed.TestRunGate_PreSnapshotFailureIsAnErrorAndDoesNotSpawn— whenStatusPorcelainfails before the gate runs, RunGate must return an error (not an outcome) and must not spawn the gate. Asserts all three: error message wraps the cause, outcome is empty,bin/gatewas not executed. Without this, the pre-snapshot failure could silently fall through to spawn.TestRunGate_PostSnapshotFailureIsBrokeContract— whenStatusPorcelainfails after a measured gate, the outcome must bebroke_contract(not an error, and not measured). Uses a call counter to fail only the secondStatusPorcelaincall. Without this, a post-snapshot failure could be returned as an error (which implies the gate never ran) or could fall through as measured.Not tested, and why:
could_not_start+ modification bypass:could_not_startreturns through theerr != nilpath fromrunGate, which exits before the post-check. This is already tested byTestRunGate_CouldNotStartIsNotDiedand adding a modification variant would testrunGate's error return, not this change's logic.broke_contract(bad envelope) + modification bypass: same structure as timeout/died — therun.Outcome != OutcomeMeasuredguard covers all non-measured outcomes uniformly. Two subcases already exercise that guard; a third adds no distinguishing power.Gate
integrationmeasuredtruemeasurement:
thresholds:
Closes #41