test: trim low-value tests, parallelize suite, and scope e2e to main#144
Merged
Conversation
A full-suite audit found ~130 cases that assert constants against their own literals, test mocks or test-local fakes instead of source, only check exit codes, or duplicate coverage that integration tests already provide. Removing them trims CI wall-time with no loss of regression protection; the e2e tier keeps only tests that exercise real shell/OS behavior.
The deleted e2e smokes were the sole coverage of info and completions status but only asserted exit codes via subprocess on every OS leg. In-process CliRunner equivalents keep the crash guard at a fraction of the cost.
The ubuntu leg duplicated ci.yml's e2e job, and the macOS/Windows legs only provide signal for shell-portability tests that rarely change in PRs. PRs keep full ubuntu e2e via ci.yml; cross-OS runs now gate main instead.
Add -n auto to addopts so every pytest invocation (local runs, the pre-commit hook, and CI) distributes tests across cores. Accepted trade-off: targeted single-test runs pay ~2s of worker startup.
wpfleger96
changed the title
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Removes about 130 tests that asserted constants against their own literal values, exercised mocks or test-local fakes instead of real code, or duplicated coverage another test tier already provided; parallelizes the remaining suite; and narrows the e2e workflow to the OS legs that exercise real cross-platform behavior.
This repository is public, so its GitHub Actions runs aren't billed, but the e2e suite still ran redundantly on every pull request — Ubuntu twice (once via
ci.yml, once viae2e.yml), plus macOS and Windows — while many unit tests carried no real regression-detection value.infoandcompletions statuswith faster in-processCliRunnertests intests/integration/test_cli.pypytest-xdistwith-n autoinaddoptsso every pytest invocation runs parallel (non-e2e suite ~26s to ~18s, e2e suite ~26s to ~6s on 4 cores)e2e.yml's macOS and Windows legs only on pushes tomain, since pull requests already get full Ubuntu e2e coverage throughci.yml