Skip to content

Corroborate Python no-test exit-code 5 - #58

Merged
hzw0813 merged 9 commits into
mainfrom
fix/unittest-exit-five-semantics
Aug 15, 2026
Merged

Corroborate Python no-test exit-code 5#58
hzw0813 merged 9 commits into
mainfrom
fix/unittest-exit-five-semantics

Conversation

@hzw0813

@hzw0813 hzw0813 commented Aug 15, 2026

Copy link
Copy Markdown
Owner

Summary

  • stop treating Python test-runner exit code 5 as automatically benign
  • require high-confidence exact zero-test observations before suppressing targeted pytest exit 5
  • require the paired targeted observer to corroborate opaque pytest/unittest exit 5
  • preserve legitimate pytest and unittest no-test behavior while failing closed when a repository-local pytest.py or unittest.py exits 5 before lifecycle observation
  • keep evidence classification and evidence-boundary explanation aligned

Why

Both pytest and the stdlib unittest CLI use exit code 5 for no-test outcomes, but the same numeric exit code is not sufficient provenance. Because Python module resolution can execute a repository-local pytest.py or unittest.py, a runner import can terminate with code 5 before ProofDiff observes any test lifecycle. The previous broad exception could therefore erase a real process failure and downgrade verification-failed to a non-failing result.

The fix accepts exit 5 only when exact high-confidence zero-test observer data corroborates the meaning. Targeted pytest can corroborate itself through its observer record; opaque pytest/unittest checks require their paired targeted observer.

Regression coverage

  • existing legitimate pytest exit-5 no-collection case remains non-failing
  • existing zero-test unittest target remains non-failing
  • repository-local pytest.py aborting with SystemExit(5) now fails closed
  • repository-local unittest.py aborting with SystemExit(5) now fails closed

Verification

  • full npm test passed
  • clean npm run clean && npm run build passed
  • git diff --check passed
  • generated dist refreshed
  • temporary updater workflows removed from the final diff

@hzw0813
hzw0813 merged commit 0dba4a6 into main Aug 15, 2026
13 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant