Severity L–M. Observed three times in one session (2026-09-07), costing one red CI
run on a pull request that was otherwise clean.
Reproduction
- Edit a
KIT_OWNED file.
- Run
uv run scripts/kit_doctor.py --generate-manifest.
- Edit a
KIT_OWNED file again — a review fix, a lint fix, anything.
- Commit and push.
test_kit_repo_self_check_is_clean then fails, naming the files whose recorded hash no
longer matches their bytes. On PR #705 this reached CI as a red run at 0b16d8b,
reporting 1 failed, 2437 passed, 3 skipped, with the failure naming
scripts/tests/test_init_sh.py and scripts/tests/test_kit_doctor.py.
Mechanism
kit-manifest.json records a content hash per kit-owned file, so it is a snapshot that
is correct only for the bytes present when --generate-manifest ran. Nothing between
that moment and the commit re-checks it. The only guard is
test_kit_repo_self_check_is_clean, which lives in the full suite — so the feedback
arrives after a make test run or a CI round trip, not at the moment the manifest went
stale.
The guard is not broken; it caught every instance. The cost is entirely in when it
reports. Note the ordering trap that makes this easy to repeat: regenerating is a natural
thing to do right after the edit that made it necessary, which is precisely when it is
most likely to be invalidated by a later edit in the same change.
Proposed fix
scripts/hooks/pre-push already exists and already refuses a push on a repository-shape
condition. Extend it: for each pushed commit touching a path in KIT_OWNED, compare that
path's hash against kit-manifest.json and refuse with the message
test_kit_repo_self_check_is_clean already prints — run kit_doctor.py --generate-manifest and commit it with your change. That moves the report from a suite
run to the push, at negligible cost.
Relationship to #47
Not a duplicate. #47 is about membership — KIT_OWNED, the manifest and the test
suite disagreeing about which files are the kit. This is about a recorded hash going
stale against content whose membership is not in question. #47's suggested CI check
would not catch it, and this hook would not catch #47.
Searched before filing: manifest stale, generate-manifest, kit_repo_self_check,
manifest pre-commit, open issues in this repo. That search turned up #47, #464,
#481, #583, #661, #662; none reports this mechanism.
Severity L–M. Observed three times in one session (2026-09-07), costing one red CI
run on a pull request that was otherwise clean.
Reproduction
KIT_OWNEDfile.uv run scripts/kit_doctor.py --generate-manifest.KIT_OWNEDfile again — a review fix, a lint fix, anything.test_kit_repo_self_check_is_cleanthen fails, naming the files whose recorded hash nolonger matches their bytes. On PR #705 this reached CI as a red run at
0b16d8b,reporting
1 failed, 2437 passed, 3 skipped, with the failure namingscripts/tests/test_init_sh.pyandscripts/tests/test_kit_doctor.py.Mechanism
kit-manifest.jsonrecords a content hash per kit-owned file, so it is a snapshot thatis correct only for the bytes present when
--generate-manifestran. Nothing betweenthat moment and the commit re-checks it. The only guard is
test_kit_repo_self_check_is_clean, which lives in the full suite — so the feedbackarrives after a
make testrun or a CI round trip, not at the moment the manifest wentstale.
The guard is not broken; it caught every instance. The cost is entirely in when it
reports. Note the ordering trap that makes this easy to repeat: regenerating is a natural
thing to do right after the edit that made it necessary, which is precisely when it is
most likely to be invalidated by a later edit in the same change.
Proposed fix
scripts/hooks/pre-pushalready exists and already refuses a push on a repository-shapecondition. Extend it: for each pushed commit touching a path in
KIT_OWNED, compare thatpath's hash against
kit-manifest.jsonand refuse with the messagetest_kit_repo_self_check_is_cleanalready prints —run kit_doctor.py --generate-manifest and commit it with your change. That moves the report from a suiterun to the push, at negligible cost.
Relationship to #47
Not a duplicate.
#47is about membership —KIT_OWNED, the manifest and the testsuite disagreeing about which files are the kit. This is about a recorded hash going
stale against content whose membership is not in question.
#47's suggested CI checkwould not catch it, and this hook would not catch
#47.Searched before filing:
manifest stale,generate-manifest,kit_repo_self_check,manifest pre-commit, open issues in this repo. That search turned up#47,#464,#481,#583,#661,#662; none reports this mechanism.