Attribute existing tests to the requirements they pin (6 → 51) - #12
Merged
Conversation
Coverage moves from 6 of 206 to 51. Nothing was implemented here: these are tests that already existed and already passed, now pointed at the statements they actually enforce, so the map reflects the compiler instead of understating it. The language chapters are largely attributed now — §12 9/17, §13 6/9, §14 9/13, §15 3/3 — mostly from facet_semantics_tests, interface_tools_tests and the error catalog matrix, which turn out to pin one normative statement each with unusual precision. §16 lands at 12/65, and the shape of what is left is the point: the covered twelve are all *validation* (rule schema, name matchers, condition typing), while the remaining fifty-three are guard *behaviour* — decision order, fail-closed timing, effect-class semantics. None of that is observable until the execution loop exists, so it is blocked on ROADMAP item 2 rather than on writing more tests. The same is true of §17 and Appendix F. Recorded in ROADMAP.md so the gap is not mistaken for laziness about test-writing. The CI baseline rises to 51 and the report is now written to docs/conformance/coverage-report.md on every run. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Second step of ROADMAP item 1.
Nothing is implemented here. These are tests that already existed and already passed, now pointed at the normative statements they actually enforce — so the coverage map reflects the compiler instead of understating it.
facet_semantics_tests,interface_tools_testsand the error-catalog matrix turned out to pin one normative statement each with unusual precision, which made the attribution mechanical rather than judgemental.What §16 tells us
The twelve covered statements are all validation — rule schema, name matchers, condition typing. The remaining fifty-three are guard behaviour: decision order, fail-closed timing, effect-class semantics. None of it is observable while
runbuilds one request and stops, so it is blocked on ROADMAP item 2 (closing the execution loop), not on writing more tests. §17 and Appendix F are in the same position.ROADMAP.mdnow says so, so the gap is not mistaken for laziness about tests.Mechanics
docs/conformance/coverage-report.mdon every run🤖 Generated with Claude Code