Check the whole forced report, and skip visibly when it cannot be checked - #16
Merged
Merged
Conversation
…cked test_the_report_is_78_columns_of_ascii stopped at the line ending in MEASUREMENT and checked only what came before it. That made its coverage depend on the environment without saying so. With torch installed the forced measurement block exists and the break skipped it; without torch the block does not exist at all. Either way the output --force uniquely produces had never been width- or ASCII-checked in any environment, and the test reported a pass in both. It now checks every line of the output, and asserts the exit code and a minimum line count first: without those, a run that produced no output at all would satisfy both loops by iterating zero times. The assertion that cannot hold without torch -- that the forced MEASUREMENT block is actually present -- moves to its own test behind an explicit skipif. A CI job that cannot run it now reports "skipped" with a reason instead of quietly passing a narrower check. Verified by forcing _torch_installed() False: 1 passed, 1 skipped, reason printed. That test uses one seed pair, because it reads line lengths rather than measuring anything, and the default 15 would cost 30 training runs to do it. What the width assertion does and does not catch, measured rather than assumed. The measurement block is printed through rich, which wraps at the console width, so an over-long sentence is wrapped before it reaches the output and cannot violate the contract. Mutations run: an unbreakable 95-character token is caught, a non-ASCII character is caught, an 80-column breakable line is NOT caught because rich wraps it first. The check is therefore load-bearing for content rich cannot wrap -- paths, hashes, long numbers -- and for ASCII. That is narrower than "the block is 78 columns" and is the honest description.
emrefbulut
added a commit
that referenced
this pull request
Sep 14, 2026
The stacked PRs #18, #19 and #20 were merged at the same moment, so only #18 reached main -- #19 merged into fix/gate-semantics-and-provenance and #20 into docs/refresh, their own bases. Everything from those two is therefore sitting on this branch and main is still on 0.4.0. That was my mistake in stacking three deep instead of retargeting; this merge is the fix. One conflict, in CHANGELOG.md. main carried the entry describing docs/release-notes/v0.5.0.md as an unpublished draft, which commit 6491f27 on this branch had already rewritten into "the four places that carry a version agree again" -- the draft label was the state this release ends. Kept this branch's side, which also carries three entries main does not have. Checked rather than assumed, because a merge across branches that both edited the test files could silently drop one side: #17's CUDA comparability tests and #16's forced-measurement width test are both present afterwards, and the suite goes from 385 to 388 passing, which is main's three additions arriving rather than anything being lost. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What changed
test_the_report_is_78_columns_of_asciinow checks every line of the--forceoutput instead of stopping at the line ending inMEASUREMENT, and asserts the exit code and a minimum line count first.The one assertion that genuinely cannot hold without torch — that the forced
MEASUREMENTblock is present — moves into its own test behind an explicitskipif.Why
The old test looked like it covered the quotable report. It did not:
Its coverage silently depended on the environment:
breakskipped itEither way the output
--forceuniquely produces had never been width- or ASCII-checked in any environment, and the test reportedpassedin both. Dumping a real run shows exactly what was escaping (lines 35–40):The exit-code and line-count assertions are there for the same class of reason: without them a run producing no output at all would satisfy every loop by iterating zero times.
How it was verified
The skip is now visible. Forcing
_torch_installed()to returnFalse— the state CI'stest (3.11)/test (3.12)jobs are in:skipped, with a reason, rather than a silently narrowedpassed.Mutations — and one of them corrected my own assumption:
The third is the honest caveat and I am not going to paper over it. The measurement block is printed through
rich, which wraps at the console width, so an over-long sentence is wrapped before it ever reaches the output and cannot violate the contract. The width assertion on that block is therefore load-bearing for content rich cannot wrap — paths, hashes, long numbers — and for ASCII. That is narrower than "this block is 78 columns", and claiming the wider thing would repeat the mistake this PR is fixing.The preflight block above it is built with
_field_linesand plainprint, so there the width check is load-bearing in full.Conventions it touches
Noticed, not fixed here
Line 32 above says
started no. This version of the command stops before trainingand lines 35–40 are the measurement it then performed. That is the self-contradiction tracked as Bölüm C item 5; this PR only stops the test from ignoring that region.