Repository navigation
Every negation in the bats suites asserts wherever it stands - #16
Merged
Merged
Conversation
`! cmd` on its own line asserts nothing in a bats body unless it happens to
be the last command of that body: bash does not apply errexit to a negated
command, so the test's verdict is the exit status of the final command and
every earlier `! cmd` is inert. shellcheck grades the two cases apart, error
for the inert ones and note for the rest, which is how these were found:
shellcheck -f gcc --shell=bash tests/*.bats | grep SC2314
Twenty-two of the thirty-one negations in this suite were inert, the largest
group in the organization. Seventeen of them are in test-ipconfig.bats, and
thirteen of those are the whole of "ip6_syntax rejects what is not an IPv6
address": only the last of the thirteen decided anything, in an IPv6
validator, in an IPv6-first distribution. The other five are
test-init-fence.bats (nat_available for a family whose nat table is missing,
and two processes that must be dead) and test-tagid.bats (an untagged page,
and the placeholder that must not survive the copy).
Every site becomes `run ! cmd`, the form that asserts wherever it stands,
including the nine that were in final position and did assert: a line whose
meaning depends on its position is the trap itself, and leaving half of them
in the old shape leaves a reader to work out which half. That form needs
bats 1.5.0, declared at the top of each file as the other suites in the
organization already declare it.
One site was a pipeline, `! printf ... | grep -q WARNING`, which `run` cannot
take; it reads the captured output of the preceding run instead.
No assertion changed its verdict: all thirteen IPv6 rejection cases reject,
and the suite is green with the assertions live, 112 tests. Coverage is
unchanged, 99.55 percent of measured lines (threshold 98).
This was referenced Sep 28, 2026
Collaborator
Author
|
Reviewed: all 31 converted refutations go red when the code under test accepts that call (the 13 ip6_syntax cases included), and none reads $output or $status from the run it replaced. One fails with status 3, from the stub, by design. No changes needed. Note: #18 and #14 still carry the old bare negations and will need the same conversion after a merge with master. |
marcos-mendez
added a commit
that referenced
this pull request
Sep 29, 2026
…-skip master gained #16, which rewrote the bare negations in tests/test-init-fence.bats. The one hunk both branches touched is the "both nat tables" test, where #16 made 'grep -q WARNING' a 'run !' and this branch had replaced it with counts of the inserts and of every prefixed journal line; the counts are kept, as review of both asked. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01SfWQScmDZ94KCS5DMYrfe6
marcos-mendez
added a commit
that referenced
this pull request
Sep 29, 2026
…script master gained #16, which turned two bare negations in the old tests/test-tagid.bats into 'run !'. This branch rewrote that file and uses 'run !' throughout, so its version is kept whole. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01SfWQScmDZ94KCS5DMYrfe6
marcos-mendez
added a commit
that referenced
this pull request
Sep 29, 2026
Every negation in the bats suites asserts wherever it stands
marcos-mendez
added a commit
that referenced
this pull request
Sep 29, 2026
…script master gained #16, which turned two bare negations in the old tests/test-tagid.bats into 'run !'. This branch rewrote that file and uses 'run !' throughout, so its version is kept whole.
marcos-mendez
added a commit
that referenced
this pull request
Sep 29, 2026
…-skip master gained #16, which rewrote the bare negations in tests/test-init-fence.bats. The one hunk both branches touched is the "both nat tables" test, where #16 made 'grep -q WARNING' a 'run !' and this branch had replaced it with counts of the inserts and of every prefixed journal line; the counts are kept, as review of both asked.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Every negation in this suite now asserts wherever it stands. Twenty-two of the thirty-one did not.
Closes #15. Part of Keel-Linux/tracker#19, which has the organization-wide sweep.
Why a bare
!is inertA bats test body has no assertions of its own: its verdict is the exit status of the last command it ran. Bash does not apply errexit to a negated command — that exemption is in the shell by design, so a script can test a command without dying — so
! cmddecides the test in final position and is inert everywhere else. The line reads as a refutation either way.shellcheck grades the two cases apart, which is what made an exact sweep possible:
error:is an inert one.note:is one in final position, which asserts today and stops asserting the moment a line is added after it.The headline: twelve of thirteen IPv6 rejection cases
tests/test-ipconfig.bats, "ip6_syntax rejects what is not an IPv6 address": thirteen cases, lines 324–336. Twelve were inert. Only1:2:3:4:5:6:7::8decided anything.Before changing them, each of the thirteen was run against
ipconfig_ip6_syntaxby hand, so that a green suite afterwards would be evidence rather than a coincidence. All thirteen are refused, and all eight of the acceptance cases are accepted. The validator is correct; it was the tests that were not running.That matters for what this PR claims: nothing shipped broken here. It is also luck, not evidence, and it is the reason the gate in
Keel-Linux/.githubexists rather than a note in a review.The rest
valid_configrefusingbogusand'',unchangedrefusing an absent stanza, the lxc build leaving noeth0,unchangedrefusing a mismatched inet6 confignat_availablefor a family with no nat table — the case the predicate exists for — and two processes that must be dead afterstop_mini_server@APP_NAME@does not survive the copy into the writable htdocsWhat changed
Every SC2314 site becomes
run ! cmd, the nine that were already in final position included, and each file declaresbats_require_minimum_version 1.5.0as the other suites in the organization do. Leaving half of them in the old shape would leave a reader to work out which half is real, and the guard reports severityerroronly, so a final-position!is not a failure — it is simply a line that will go inert when somebody appends to the body.One site was a pipeline and could not be prefixed:
runtakes a command, not a pipeline. The here-string is expanded beforerunexecutes, so it reads the output ofrun "$FENCE" startabove it.Test plan
The thirteen IPv6 rejection cases verified against the validator by hand, before the conversion. All refused.
shellcheck -f gcc --shell=bash --severity=error --include=SC2314 tests/*.batsreports nothing.Each converted assertion proven to bite: replacing one case with a valid address fails the test at that line.
bats tests/green, 112 tests. No assertion changed its verdict.tests/coverage.sh: 99.55 percent of measured lines, threshold 98. Unchanged.Tests ship nothing, so
package / changelogpasses by reporting that nothing that ships changed. No changelog entry and no version bump.