Repository navigation
fix: run the suite against the installed wheel, not the source tree - #159
Merged
Merged
Conversation
|
Important
This repository does not receive automatic reviews because it has fewer than 10 stars. ⚙️ Run configurationConfiguration used: defaults Review profile: CHILL Plan: Advanced Run ID: Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
This was referenced Sep 26, 2026
The Run Tests step called `python -m pytest`. The -m form puts the working directory first on sys.path, and the working directory is the repository root. For a flat-layout package the root holds the package directory, so the suite imported the source tree and never the wheel the job had just built and installed. A packaging defect passed: a module left out of the wheel, a dependency not declared, package data not shipped. Use the console script, which adds no such sys.path entry. The working directory stays the repository root, so a suite that reads a repo-relative fixture path is unaffected, and prepend import mode still puts the test directory on sys.path, so a helper module beside the tests still resolves. The console script is necessary and not sufficient, and the comment above the command now says so. Prepend import mode inserts the first directory above the test file that is not a package, so a test/__init__.py or a conftest.py at the repository root makes that directory the repository root again and the shadowing returns. Of 413 flat-layout callers across the five orgs, 228 get the fix and 185 keep testing their checkout; the remedy there is per repository, either removing the test package's __init__.py or `--import-mode=importlib` in pytest_args. T-5492 carries that set, and T-5337 the root-conftest part of it. The comment also notes that the console script resolves through PATH where the -m form bound the interpreter, which matters only if a pre_test_command ever activates a venv. test_build_tests_runs_the_wheel.py reads the workflow and adds eight controls that run pytest over a fixture whose installed copy and source copy disagree, one per property: the old command imported the source, the new command imports the installed copy, a tests-local helper import still resolves, a repo-relative fixture path still resolves, a test package puts the rootdir back, a root conftest.py does the same, and --import-mode=importlib holds in both of those shapes. The two shape controls assert the failure names the source copy, so a pytest that fails to start cannot pass them. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
JarbasAl
force-pushed
the
ci/build-tests-runs-the-wheel
branch
from
September 26, 2026 08:40
32e3751 to
a429cee
Compare
JarbasAl
marked this pull request as ready for review
September 26, 2026 10:20
This was referenced Sep 27, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
The defect
The
Run Testsstep ranpython -m pytest. The-mform puts the workingdirectory first on
sys.path, and the working directory is the repository root.For a flat-layout package the root holds the package directory, so
import mypkgresolved to the source tree and never to
site-packages. The job built a wheel,installed it, and then tested the checkout instead. A packaging defect passed in
every caller: a module left out of the wheel, a dependency not declared, package
data not shipped.
The proof
A control package whose wheel contains no code at all:
The old command and the new one, same tree, same venv:
The reverse control, a correctly packaged repository with a test that asserts
where the import came from:
The fix
One token:
pytest, notpython -m pytest. The console script adds no workingdirectory entry.
The working directory is deliberately not changed. A suite that reads a
repo-relative fixture path keeps working, and prepend import mode still puts the
test directory on
sys.path, so a helper module beside the tests still resolves.Both are controls in the new test file.
--import-mode=importlibwas measured and is not needed: dropping-missufficient, and importlib mode would stop a tests-local helper import from
resolving.
Blast radius, counted
A screen for callers whose tests import a root-level name that the packaging
config never names, which is where a new red is possible:
The screen is a screen: a name that appears anywhere in the packaging config
counts as declared, so it can miss a case and it can raise a false one. It is
not a substitute for the sweep.
A limit worth naming
48 of the 182 caller test directories call
sys.paththemselves. Where aconftest re-adds the repository root, this fix is defeated and the suite still
tests the source tree. Those repositories need the
sys.pathline removed, oneat a time, and that is separate work.
Canary runs
Three callers were pointed at this branch. One produced the evidence; the other
two are red for reasons that predate this change, and both are named below as
fleet work, not as a result of this one.
OpenVoiceOS/nebulento#57, run 36218046118: green on all five versions.
The executed step, read from the log of job 108337853495:
The new command form ran, the rootdir is still the repository root, and a real
178-test suite stayed green. nebulento is flat layout and carries a second root
package (
benchmark), so it is the shape most at risk.TigreGotico/linux2mqtt#15: no evidence, the repository has no working CI.
7 of its 9 workflows that call a gh-automations reusable end in
startup_failure, and the 2 that call none (docker.yml,conventional-label.yml) run. The same is true on its other branches, so itpredates this change. The cause is not this workflow: the repository default is
readand those 7 callers carry nopermissions:block, so the forge refusesthe run before any job starts. Diagnosed and fixed in linux2mqtt#16.
VoiceGamez/ovos-skill-moon-game#21: no evidence, red before the tests.
The
Install Built Packagestep fails to buildfann2==1.0.7(ld: cannot find -ldoublefann) on all five versions, so no job reachesRun Tests. Itsbuild-tests has failed this way since 2026-06. Filed separately.
That leaves the
__init__prediction in the screen above unverified by arun: moon-game was the one repository that could have shown it, and it never
gets far enough to import anything.
Local test result
The 6 are pre-existing and local: no
actionlintbinary and noovos-plugin-managerin this environment.Mutation check: restoring
python -mmakestest_the_workflow_does_not_invoke_pytest_through_dash_mfail, so the guard isnot a silent pass.
🤖 Generated with Claude Code