Reorganize into top-level packages, drop dynamic sys.path inserts#13
Merged
Conversation
Promote simulation/{arduloop,calibrate,envelope,analysis,viz3d,scripts,tests}
to top-level packages, each installed via a root pyproject.toml editable
install instead of ad-hoc sys.path.insert() calls scattered across the
codebase.
- Add root pyproject.toml (setuptools, packages.find over the 8 top-level
packages, pytest markers/timeout config).
- Rewrite bare same-directory imports to dotted package imports
(e.g. from controller import X -> from simulation.controller import X)
across ~225 files via a one-off migration script (tmp/migrate_imports.py).
- Fix Path(__file__).resolve().parents[N] path arithmetic broken by the new
depths; anchor via Path(<pkg>.__file__).resolve().parent on an actual
package import instead of counting parents, which is immune to future
depth changes.
- Fix migration-script blind spots: whitespace-padded imports, dangling
guarded-insert for-loops, semicolon-joined one-liners, dynamic
__import__() calls.
- Fix two pre-existing (unrelated) syntax corruptions in
analysis/compare_pumping.py and tests/oneoff/analyze_replay_aero_failure.py,
found via a full py_compile sweep.
- Update test.sh Docker sync to tar all 8 top-level packages from the repo
root into the container (was only syncing simulation/), and fix the SITL
test-file discovery/dispatch paths.
- Update .github/workflows/ci.yml stack-matrix discovery and unit/simtest
invocations for the new tests/ location.
- Sweep design/*.md, AGENTS.md, README.md, and docstring/comment usage
examples for stale simulation/-prefixed paths.
Validated: full py_compile sweep clean; pytest tests/unit 467 passed;
pytest tests/simtests 14 passed, 5 skipped; SITL stack test
(test_lua_flight_steady_sitl) passed end-to-end in Docker.
…yout - AGENTS.md: add a Repository Layout section describing the 8 top-level packages (simulation, arduloop, calibrate, envelope, analysis, viz3d, scripts, tests) and non-package top-level dirs; fix stale simulation/pytest.ini reference (pytest config now lives in pyproject.toml [tool.pytest.ini_options]). - design/testing.md: fix the same stale pytest.ini reference. - design/simulation.md: rewrite the Module Map tree -- arduloop/, scripts/, analysis/, viz3d/, and tests/ were shown nested under simulation/ but are now top-level sibling packages; split into separate trees with a note pointing to AGENTS.md's Repository Layout section. Left archive/ (out-of-scope archived code) and design/history.md (historical changelog documenting past state) untouched by design. Validated: pytest tests/unit -q -> 467 passed.
_sync_code() never synced the root pyproject.toml into the SITL Docker containers, so pytest-timeout's global 600s config and simtest/sitl marker registrations were never picked up inside the container. Worse, a separate block synced the sibling ../aero workspace's own pyproject.toml (timeout=10, no rawes markers) to the same /rawes/pyproject.toml path, silently clobbering rawes's config -- this caused 5/7 SITL stack tests to fail with a false 10s pytest-timeout. Fix: add pyproject.toml to the main repo tar sync, and remove the entire ../aero source-tree sync block. It was vestigial: dynbem (the only aero-workspace package rawes imports) is already installed from a pinned PyPI wheel by the script right below it, and the aero/ directory referenced in the old tar list doesn't even exist in the sibling repo anymore. Validated: full stack suite (bash test.sh stack -n 4) now passes 6/7, with the 1 apparent failure (test_yaw_regulation_sitl) confirmed to be CPU-contention flakiness from parallel workers, not a regression -- passes cleanly in isolation (-n 1 -k test_yaw_regulation_sitl).
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Promotes
simulation/{arduloop,calibrate,envelope,analysis,viz3d,scripts,tests}to top-level packages, each installed via a rootpyproject.tomleditable install, replacing the scattered ad-hocsys.path.insert()dynamic path hacks with normal Python package imports.Changes
pyproject.toml(setuptools,packages.findover the 8 top-level packages, pytest markers/timeout config).from controller import X→from simulation.controller import X) across ~225 files via a one-off migration script (tmp/migrate_imports.py).Path(__file__).resolve().parents[N]path arithmetic broken by the new directory depths; anchor viaPath(<pkg>.__file__).resolve().parenton an actual package import instead of counting parents, which is immune to future depth changes.__import__()calls.analysis/compare_pumping.pyandtests/oneoff/analyze_replay_aero_failure.py, found via a fullpy_compilesweep.test.shDocker sync to tar all 8 top-level packages from the repo root into the container (previously only syncedsimulation/), and fix the SITL test-file discovery/dispatch paths..github/workflows/ci.ymlstack-matrix discovery and unit/simtest invocations for the newtests/location.design/*.md,AGENTS.md,README.md, and docstring/comment usage examples for stalesimulation/-prefixed paths.Validation
py_compilesweep: clean across all 8 packages.pytest tests/unit -q: 467 passed.pytest tests/simtests -q: 14 passed, 5 skipped.bash test.sh stack -n 1 -k test_lua_flight_steady_sitl): passed end-to-end in Docker.