Skip to content

A negation in tests/postfix-local.bats asserts nothing, and COVERAGE.md is wrong about which ones do #9

Description

@marcos-mendez

A bats test body has no assertions of its own: its verdict is the exit status of the last command it ran. Bash does not apply errexit to a negated command, so ! cmd on its own line decides the test when it is the final command of the body and is inert everywhere else.

shellcheck -f gcc --shell=bash tests/*.bats | grep SC2314

tests/postfix-local.bats has three bare negations. One is inert, line 96, in "stops at the first failing postconf":

@test "stops at the first failing postconf" {
    export STUB_POSTCONF_FAIL=myhostname
    run "$SCRIPT"
    [ "$status" -eq 1 ]
    logged "postconf -e myhostname=mail.example.com"
    ! logged smtpd_banner        # line 96 — inert
    ! logged postmulti           # line 97 — last, asserts
}

The test is named for stopping at the first failure, and the assertion that the banner was never set is the one that says it stopped. That one did not run. A script that carried on past the failing postconf and set the banner anyway would pass this test.

Lines 66 and 97 are last in their bodies and do assert. That distinction is worth stating because COVERAGE.md on fix/keel-apt-identity (#8) says all three are inert, which is wrong about two of the three, and that text needs correcting wherever it lands.

What to do

Write every negation run ! cmd, with bats_require_minimum_version 1.5.0, which asserts wherever it stands — the two in final position included. A file that mixes the two forms leaves a reader to work out which half is real, and adding one line after a final-position ! silently disables it.

Test plan

  • shellcheck --severity=error --include=SC2314 over tests/*.bats reports nothing.
  • bats tests/postfix-local.bats green with the inert assertion live.
  • tests/coverage.sh at the committed threshold of 100.
  • Tests and COVERAGE.md ship nothing, so package / changelog passes by reporting that nothing that ships changed.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions