Repository navigation
fix(check): honor bin/check's documented exit-75/79 grading contract - #83
developerz-ai[bot] wants to merge 1 commit into
Conversation
bin/check was three bare cargo commands under set -euo pipefail, so every outcome was cargo's own status: an unmet environment was indistinguishable from a red change, and a partially graded gate from a green one. The verify-change skill reserves 75 (a resource the gate needs is unreachable) and 79 (everything that ran passed, part of the grade is deferred to CI). Guard preconditions first and exit 75 when cargo, rustfmt or clippy is missing; run the required checks (fmt, clippy, test) plus CI's shellcheck and cargo audit, skipping the two loudly when their tool is not installed; exit 1 when anything ran and failed, 79 when nothing failed but a gate was deferred, 0 only when everything ran and passed. No check is removed, scoped down or silenced. The pre-commit/pre-push hooks now translate that surface: 0 and 79 let the commit or push through, 1 and 75 block with the right blame. Regression tests drive the real script in a sandbox with a configurable stub cargo and a deterministic restricted PATH, covering success, genuine failure, unmet environment, partial grading and subprocess status propagation. Dz-Task-Id: tsk_c2e9755ddeb79dd3e995eaa912acd5f8
|
Make bin/check honor its documented grading-status contract
Work landed on #83 — watching it toward merge. 20m · 155 tool calls · 17 messages |
🟡 Reviewed1 actionable comment(s) · grounded on your code |
There was a problem hiding this comment.
Review summary — 7 file(s), 1 finding(s).
Critical 0 · Major 1 · Minor 0 · Nit 0
Adds five-check grading, exit-code translation in hooks, and sandbox regression tests. The required-tool guard still misclassifies missing rustup components as failed changes.
- test files were not shown to the reviewer — this diff touched no path recognised as a test, so no assertion was available to read the change against
| if command -v "$1" >/dev/null 2>&1; then | ||
| return 0 |
There was a problem hiding this comment.
major · bug With rustup installed but rustfmt or clippy absent from the selected toolchain, command -v finds rustup's proxies, so the guard passes and the gate returns 1 instead of 75. Probe the selected tools' availability before grading and add a regression case with present proxies but missing components.
There was a problem hiding this comment.
Not changing this: The required fix belongs in bin/check:19-28: its command -v-only prerequisite check accepts unusable rustup proxies, and a simulated proxy run reaches grading and exits 1. This round's enforced write lock refuses edits to bin/check, so neither the implementation nor a passing regression test can be delivered.
There was a problem hiding this comment.
The decline does not hold at 8006fc718b6b: the finding still stands at bin/check:19. The prerequisite guard still uses only command -v, so present rustup proxies pass even when the selected toolchain lacks rustfmt or clippy. Their later failures are recorded as failed checks and exit 1 rather than 75; no component-availability probe or proxy regression case was added.
There was a problem hiding this comment.
Not changing this: The finding is real and I reproduced it (with rustup-style rustfmt/cargo-clippy proxies on PATH that exit non-zero, bin/check's guard at bin/check:19 command -v "$1" passes and the gate exits 1, not 75). The only fix is in bin/check itself — probe the tools instead of resolving them at bin/check:18-28 — but that file is write-locked this round: code_edit on bin/check is refused with 'bin/check is the repository's own gate (main already has it) … the land refuses any change to it', even after widen_file_set (PRRT_kwDOTGUmXc65XE7L) returned granted. tests/dev_scripts.rs (in my set) can only add the regression case, which would fail against the unchanged guard and turn the gate red. The fix needs a separate pull request that a person opens against bin/check.
There was a problem hiding this comment.
The decline does not hold at 8006fc718b6b: the finding still stands at bin/check:19. The prerequisite guard still checks only command -v, so rustup proxies pass even when the selected toolchain lacks rustfmt or clippy. Subsequent component failures are recorded as failed checks and exit 1 rather than 75; no availability probe or proxy regression test was added.
There was a problem hiding this comment.
Not changing this: The defect remains at bin/check:19, where the required-tool guard uses command -v; correcting it requires changing bin/check, but this run explicitly refuses writes to that locked gate file.
There was a problem hiding this comment.
Not changing this: The finding is valid: bin/check:19-28 only uses command -v, which accepts rustup proxies without verifying the selected rustfmt/clippy components. This run obtained widening authorization, but the platform's write lock then refused the required bin/check edit; no other file can safely correct the gate's exit-75 behavior.
There was a problem hiding this comment.
Not changing this: The required implementation is bin/check:18-28, whose command -v guard accepts rustup proxies without selected components. I requested widening and it was granted, but the harness then refused the bin/check edit as write-locked; no source change can correct this behavior in this round.
There was a problem hiding this comment.
Not changing this: The finding is valid: bin/check:19-28 only uses command -v, which accepts rustup proxy binaries without their selected components. The required fix is in bin/check, but this round's platform write lock explicitly refuses changes to that file even after widen_file_set returned it to the existing set.
There was a problem hiding this comment.
Not changing this: Confirmed: bin/check:18-28 checks only command -v, which accepts rustup proxies even when the selected toolchain lacks rustfmt or clippy; failures then reach the exit-1 grading path at bin/check:61-70. The required fix is in bin/check, explicitly write-locked by this round's brief; it must be unlocked or changed in a separate permitted PR, together with a present-proxy/missing-component regression in tests/dev_scripts.rs.
|
No code change warranted. The Declined PRRT_kwDOTGUmXc65XE7L because its required implementation file, |
There was a problem hiding this comment.
Review summary — 7 file(s), 1 finding(s).
Critical 0 · Major 1 · Minor 0 · Nit 0
Adds five-check grading, exit-code translation in hooks, and sandbox regression tests. The required-tool guard still misclassifies missing rustup components as failed changes.
- test files were not shown to the reviewer — this diff touched no path recognised as a test, so no assertion was available to read the change against
|
A fix is queued and waiting for a machine that can run this repository. Nothing is needed from you right now.
Status for head |
|
No code change warranted. The No change this round — the fix is refusedThread PRRT_kwDOTGUmXc65XE7L — declined (refused widening / write lock)Finding (major): with rustup installed but rustfmt or clippy absent from the Reproduced, not merely read. Probe: a stub The guard at bin/check:19 ( Why no file changedThe fix is a one-line-per-tool change inside bin/check itself: probe the tool with
The rest of the finding — "add a regression case with present proxies but No other file this pull request touches carries the defect class: the only other What the gate needsA separate, human-opened pull request against |
There was a problem hiding this comment.
Review summary — 7 file(s), 1 finding(s).
Critical 0 · Major 1 · Minor 0 · Nit 0
Adds five-check grading, exit-code translation in hooks, and sandbox regression tests. The required-tool guard still misclassifies missing rustup components as failed changes.
- test files were not shown to the reviewer — this diff touched no path recognised as a test, so no assertion was available to read the change against
|
No code change warranted. The Unable to address PRRT_kwDOTGUmXc65XE7L: the required implementation is in bin/check:19-28, but that file is write-locked for this lane. The harness explicitly refused the attempted edit. The current command -v precondition accepts rustup proxy shims despite missing selected-toolchain components; tests/dev_scripts.rs needs the requested regression only alongside a permitted bin/check fix, because the regression correctly fails beforehand. |
|
No code change warranted. The No code change: the review finding requires changing |
|
No code change warranted. The No code changed. The only review finding requires changing |
|
No code change warranted. The No code changed. The review finding requires a component-availability probe in |
|
No code change warranted. The Unable to address PRRT_kwDOTGUmXc65XE7L: its root cause is |
|
No code change warranted. The Unable to fix PRRT_kwDOTGUmXc65XE7L: bin/check:18-28 needs a selected-tool availability probe (for example, invoke Targeted verification was attempted but the environment could not start Cargo: |
|
No code change warranted. The Review thread PRRT_kwDOTGUmXc65XE7L remains blocked by the repository gate lock. Permission to modify Attempted targeted verification was blocked by the environment before compiling the test: |
|
No code change warranted. The Unable to fix review thread PRRT_kwDOTGUmXc65XE7L: the necessary change is to bin/check:19-28 so its preflight probes rustfmt and cargo-clippy rather than merely locating rustup proxy binaries. Direct edit was refused by the platform's bin/check write lock after widen_file_set reported the path was already in the exclusive set. I added the regression coverage in tests/dev_scripts.rs, but cannot leave it as a shipping change while bin/check remains locked and unfixed. |
|
No code change warranted. The Blocked: the sole review finding requires changing Confirmed root cause: Required follow-up: permit an ordinary coding change to |
What
bin/check was three bare cargo commands under set -euo pipefail, so every
outcome was cargo's own status: an unmet environment was indistinguishable
from a red change, and a partially graded gate from a green one. The
verify-change skill reserves 75 (a resource the gate needs is unreachable)
and 79 (everything that ran passed, part of the grade is deferred to CI).
Guard preconditions first and exit 75 when cargo, rustfmt or clippy is
missing; run the required checks (fmt, clippy, test) plus CI's shellcheck
and cargo audit, skipping the two loudly when their tool is not installed;
exit 1 when anything ran and failed, 79 when nothing failed but a gate was
deferred, 0 only when everything ran and passed. No check is removed,
scoped down or silenced.
The pre-commit/pre-push hooks now translate that surface: 0 and 79 let the
commit or push through, 1 and 75 block with the right blame. Regression
tests drive the real script in a sandbox with a configurable stub cargo and
a deterministic restricted PATH, covering success, genuine failure, unmet
environment, partial grading and subprocess status propagation.
Acceptance
bin/checkexits 0 only when fmt, clippy, test, shellcheck and cargo audit all ran and passedbin/checkexits 1 when any check that ran failed, naming the failed check, and still exits 1 when a present shellcheck/cargo-audit failsbin/checkexits 75 with an actionable diagnostic naming the missing tool when cargo, rustfmt or clippy is unavailablebin/checkexits 79 with a named SKIP line when shellcheck and/or cargo-audit is not installed and every required check passedcargo test --locked --test dev_scriptscovers success, genuine failure, unmet environment, partial grading and subprocess status propagationbin/checkstill cd's to the repo root before invoking cargo (tests/dev_scripts.rs cwd assertions stay green)Changes
bin/check(edit) — Implement the SKILL.md:29-46 exit contract over the existing gate. Keepcd "$(dirname "$0")/.."(tests/dev_scripts.rs:116-128 pins it) and keep the three required checks verbatim (fmt --check, clippy --locked --all-targets -- -D warnings, test --locked) plus CI's other two (ci.yml:57cargo audit, ci.yml:66-67shellcheck bin/* deploy/install.sh deploy/debian/postinst .githooks/pre-commit .claude/hooks/*.sh .github/scripts/*.sh). Precondition guard FIRST: missing cargo/rustfmt/clippy -> exit 75 with an actionable line naming the missing tool and the fix and saying the environment [trimmed]tests/dev_scripts.rs(edit) — Regression tests for the new contract, reusing the existing Sandbox: extend STUB_CARGO (:29-38) so it honours env knobs (per-subcommand exit code, e.g. CARGO_STUB_FAIL=test/clippy/fmt, and pass-through of the requested args) and add a restricted-PATH run helper for the check tests: PATH = stub dir : a toolbox dir of symlinks (dirname, plus cargo-audit/shellcheck stubs only when a test wants them), so tool presence is deterministic regardless of the host. Add tests for: exit 0 when everything runs green; exit 1 when cargo fmt or cargo test fails and when a present shellcheck/audit fai [trimmed].githooks/pre-commit(edit) — Consume the new status surface: replace the bareexec bin/check(:7) withstatus=0; bin/check || status=$?and a case that exits 0 on both 0 and 79 (79 means all graded checks were green and CI grades the rest — blocking it would refuse a correct change) while propagating 1 and 75 unchanged, with a one-line message distinguishing 75 (blame the environment) from 1 (blame the change). Keep set -euo pipefail-safe form; keep shellcheck-clean..claude/hooks/pre-commit.sh(edit) — Same status translation as .githooks/pre-commit (:7exec bin/check): exit 0 on gate 0 or 79, propagate 1 and 75. Preserve the existing core.hooksPath short-circuit (:6) so an active git hook does not run the suite twice..claude/hooks/pre-push.sh(edit) — Same status translation as the other hooks (:7exec bin/check): exit 0 on gate 0 or 79 (79 explicitly permits the push — SKILL.md:39), propagate 1 and 75. Preserve the core.hooksPath short-circuit (:6)..claude/skills/verify-change/SKILL.md(edit) — Make the contract concrete instead of aspirational: in the exit-codes section (:29-46) name the exact cases bin/check now maps — 75 for a missing cargo/rustfmt/clippy, 79 for shellcheck or cargo-audit being uninstalled while fmt/clippy/test passed, 1 for any check that ran and failed — and state that a 79 must be committed and pushed (the hooks treat it as continue) while a 75 blocks. Update theproverow (:20) and the gate claim (:13) to the real command list (fmt, clippy, test, shellcheck, cargo audit; the last two skip loudly). Do not change the meaning of any code.CLAUDE.md(edit) — The command-table row (:46) describes bin/check as three cargo commands; bring it in line with the script and with CI (fmt --check, clippy --locked --all-targets -D warnings, cargo test --locked, shellcheck, cargo audit) and state the exit contract in one line (0 pass, 75 unmet environment, 79 partially graded, other non-zero red). Surgical edit of that row only; leave :54 and the rest untouched.Verification
./bin/checkpassed (exit 0) in 42s.What the editors ran to see the change working, and what they saw:
shellcheck bin/check .githooks/pre-commit .claude/hooks/pre-commit.sh .claude/hooks/pre-push.sh && rustup run 1.100.0 cargo test --locked --test dev_scripts(gate): shellcheck exited 0 (no output); test result: ok. 17 passed; 0 failed — covering green (0), each failing required check (1), present-but-failing shellcheck and cargo audit (1), each missing optional tool (79, with SKIP lines), each missing required tool (75, nothing run), and hook propagation of 0/1/75 plus the core.hooksPath short-circuitenv PATH=<stub-only dir with cargo/rustfmt/cargo-clippy/bash/dirname> <sandbox>/bin/check (no shellcheck, no cargo-audit)(gate):SKIP: cargo audit — cargo-audit is not installed here; CI runs it (.github/workflows/ci.yml), same for shellcheck, thenbin/check: exiting 79 — every check that ran passed; CI grades: cargo audit shellcheck; EXIT=79same stub PATH with green shellcheck + cargo-audit stubs, then with a failing shellcheck stub(gate): green:bin/check: all checks passed, EXIT=0; failing shellcheck:bin/check: FAILED: shellcheck — the change is red, EXIT=1PATH=<dir with cargo, rustfmt, bash, dirname but no cargo-clippy>; bin/check(gate):bin/check: cannot grade — cargo-clippy is not installed.\n fix: rustup component add clippyandbin/check: exiting 75 — a tool the gate needs is missing; the environment is at fault, not the change.; EXIT=75rustup run 1.100.0 rustfmt --check --edition 2024 tests/dev_scripts.rs(gate): EXIT=0 (one tuple-formatting diff found and fixed before this run)code_read bin/check(docs): Guard blockrequire cargo/rustfmt/cargo-clippy→exit 75; thenrun fmt,run clippy,run test,run_optional cargo-audit,run_optional shellcheckover bin/* deploy/install.sh deploy/debian/postinst .githooks/pre-commit .claude/hooks/.sh .github/scripts/.sh; thenexit 1on any failed check,exit 79when only skips with SKIP: lines. Docs now quote exactly this list and contract.code_grep 'verify-change|SKILL|75|79' in tests/dev_scripts.rs(docs): Test doc pins the same contract: '0 green, 1 a check that ran failed, 75 a required tool is missing, 79 every check that ran passed but an optional CI gate could not run' plus 'hooks exit 0 on 0 and 79, and propagate 1 and 75' — the SKILL text now states the same four codes and the same hook rule.code_read bin/check (full file, 78 lines)(docs): script runscargo fmt --check,cargo clippy --locked --all-targets -- -D warnings,cargo test --locked, then run_optional cargo-audit/shellcheck; requires cargo/rustfmt/cargo-clippy first and exits 75; skips print "SKIP: — is not installed here"; exits 1 on any failed check, else 79 when skipped[] is non-empty, else 0code_changes CLAUDE.md .claude/skills/verify-change/SKILL.md(docs): CLAUDE.md:46 now lists all five CI checks with the 0/75/79/other-non-zero contract; SKILL.md:13 gate claim and :23 prove row list the same five checks, :41-43 exit table maps 75 to missing cargo/rustfmt/clippy and 79 to uninstalled shellcheck/cargo-audit, :21 install row notes the 75 guard covers rustfmt and clippyNotes
Handoffs this run could not complete:
CLAUDE.md— bin/check now runs CI's full five-check list (fmt, clippy, test, cargo audit, shellcheck) and owns an exit contract (0/1/75/79) that the three hooks translate. CLAUDE.md:46 still describes the gate as three commands only.; The Commands table rowFull gate (fmt --check + clippy --locked --all-targets -D warnings + cargo test --locked; e2e + dev_scripts included)must also namecargo auditandshellcheck bin/* deploy/install.sh deploy/debian/postinst .githooks/pre-commit .claude/hooks/*.sh .github/scripts/*.shand state the exit contract (0 green; 79 = optional gate absent, CI grades it; 75 = required tool missing; 1 = red) so the documented command list matches what the script implements. (the editor sent to make it neither wrote it nor said why not)Raised automatically by a developerz.ai maintainer lane.