docs(harness): record the fixture-vacuity case in the anti-vacuous doctrine - #292
Merged
Merged
Conversation
…ctrine `coverage.mjs` states the rule for guards: zero scanned is a failure, because a check that scanned nothing did not run. Nothing states it for the guard's TEST, and the failure has the same shape one level up — a test whose fixture never reaches the branch under test passes, and reads as proof. Observed on 2026-07-30 (GT-633) while fixing two parser defects in `41-validate-evidence-commands`. Six tests were written for the fix; three of them asserted that a command is NOT reported dead, and passed against the UNFIXED parser too, because the branch that reports it dead is guarded by `pkg?.scripts` and the fixture's root package.json declared no `scripts`. The assertions were true, the fix was real, and those tests proved nothing about it. The rule now written down next to its sibling: when a test is written to pin a fix, RUN IT AGAINST THE UNFIXED CODE AND WATCH IT FAIL. A test that passes before the fix is the vacuous pass this file already describes, wearing a green tick from the test runner instead of from a guard. Its other half is recorded too, because it is the part that gets skipped: making the fixture live took those six from 3-of-6 red to 5-of-6 red, and the sixth passes either way. That one is kept and LABELLED a regression guard rather than counted as proof — when a test cannot be made to fail, say so in the test instead of letting the count imply coverage it does not have. This is doctrine for whoever writes the next guard, which is why it lives here rather than in the GT-633 board entry: a gap row is read once, this file is read by everyone who reaches for `assertScanned`. Verified: coverage.test.mjs 12/12; the module still loads and exports ZeroCoverageError, assertScanned, assertScannedPerSource, scanned; 42-validate-guard-denominators green over 60 guards; 01-validate-docs green. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
|
You have reached your Codex usage limits for code reviews. You can see your limits in the Codex usage dashboard. |
📊 Bilingual Coverage ImpactPR Changes
Repository Coverage
✅ Good: All EN changes have ES counterparts. Generated by GitHub Actions |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Pull Request Summary
coverage.mjsstates the rule for guards: zero scanned is a failure, because a check that scanned nothing did not run. Nothing stated it for the guard's test, and the failure has the same shape one level up — a test whose fixture never reaches the branch under test passes, and reads as proof.25 lines of doctrine, no behaviour change.
The case it records
Observed 2026-07-30 (GT-633) while fixing two parser defects in
41-validate-evidence-commands. Six tests were written for the fix. Three of them passed against the unfixed parser too — they asserted that a command is not reported dead, and the branch that reports it dead is guarded bypkg?.scripts, while the fixture's rootpackage.jsondeclared noscriptsat all. The assertions were true, the fix was real, and those tests proved nothing about it.The rule, now written next to its sibling:
And the half that usually gets skipped
Making the fixture live took those six from 3-of-6 red to 5-of-6 red against the old parser. The sixth passes either way — it is kept and labelled a regression guard, not counted as proof. When a test cannot be made to fail, say so in the test rather than letting the count imply coverage it does not have.
Why here and not on the board
GT-633 is already registered on
main(c3221276) and this lesson is not really about that gap — it is doctrine for whoever writes the next guard. A gap row is read once; this file is read by everyone who reaches forassertScanned.Verification
coverage.test.mjs— 12/12ZeroCoverageError,assertScanned,assertScannedPerSource,scanned42-validate-guard-denominators.mjs— green over 60 guards01-validate-docs.mjs— green, 1491 filesEvolith Core Quality Gates
.mdtouched — the change is a comment block in a.mjsmodule.01-validate-docs.mjsrun and green.Linked ADRs / Issues
GT-557's anti-vacuous rule, enforced byGT-578; the case comes fromGT-633.Conventional Commits
docs:🤖 Generated with Claude Code