Summary
The CI test runner supports exactly one test convention: a tests/ (plural)
directory, test_*.py (prefix) filenames, and Python's unittest loader.
Issue #52 made discovery recursive and added a per-addon import root so
nested-package addons that follow this convention (e.g. NameSuite, upstream
PR 941 — 20 modules / 234 tests now run) work. But a second in-flight
submission uses a different convention, which #52 deliberately did not
broaden to (narrow-scope decision). This issue records that gap so we can decide
it on its own.
The divergence (dsblank PR 1)
dsblank/addons-source PR 1 ("Add FastFiltersLib, SQLiteFastFilters,
PostgreSQLFastFilters") ships its tests as:
test/ (singular) directory — not tests/.
*_test.py (suffix) filenames — e.g. filter_overrides_test.py,
sql_compat_test.py — not test_*.py.
- pytest, via
conftest.py that does the sys.path setup, including
cross-addon sharing (SQLite/PostgreSQL conftest.py add FastFiltersLib/
and FastFiltersLib/test/ to the path to reuse shared base classes), and a
sibling import from filter_overrides_base import ….
Consequences for our runner:
- Discovery (
*/tests/**/test_*.py) does not match test/ + *_test.py,
so these suites are never found.
- Even if found, the
unittest loader does not execute conftest.py, so
the sys.path setup (and the cross-addon sharing) would not run — from fastfilterslib… import / from filter_overrides_base import would fail.
Decision needed
Pick one (or a mix):
- Require addons to conform to the documented convention (
tests/ +
test_*.py + unittest, with per-addon import root and relative imports for
private helpers). Lowest CI cost; pushes the work to each addon.
- Broaden discovery to also accept
test/ and *_test.py (still unittest).
Cheap glob change, but does not solve pytest conftest.py / cross-addon
sharing — partial at best.
- Add pytest as a CI runner so
conftest.py (fixtures, cross-addon path
sharing) works. Largest change — effectively a second test-execution mode.
This needs a maintainer convention call (it touches every future addon and the
two core maintainers' differing styles), so it is intentionally not bundled
into #52 / PR 820.
References
Summary
The CI test runner supports exactly one test convention: a
tests/(plural)directory,
test_*.py(prefix) filenames, and Python'sunittestloader.Issue #52 made discovery recursive and added a per-addon import root so
nested-package addons that follow this convention (e.g. NameSuite, upstream
PR 941 — 20 modules / 234 tests now run) work. But a second in-flight
submission uses a different convention, which #52 deliberately did not
broaden to (narrow-scope decision). This issue records that gap so we can decide
it on its own.
The divergence (dsblank PR 1)
dsblank/addons-source PR 1 ("Add FastFiltersLib, SQLiteFastFilters,
PostgreSQLFastFilters") ships its tests as:
test/(singular) directory — nottests/.*_test.py(suffix) filenames — e.g.filter_overrides_test.py,sql_compat_test.py— nottest_*.py.conftest.pythat does thesys.pathsetup, includingcross-addon sharing (SQLite/PostgreSQL
conftest.pyaddFastFiltersLib/and
FastFiltersLib/test/to the path to reuse shared base classes), and asibling import
from filter_overrides_base import ….Consequences for our runner:
*/tests/**/test_*.py) does not matchtest/+*_test.py,so these suites are never found.
unittestloader does not executeconftest.py, sothe
sys.pathsetup (and the cross-addon sharing) would not run —from fastfilterslib… import/from filter_overrides_base importwould fail.Decision needed
Pick one (or a mix):
tests/+test_*.py+ unittest, with per-addon import root and relative imports forprivate helpers). Lowest CI cost; pushes the work to each addon.
test/and*_test.py(still unittest).Cheap glob change, but does not solve pytest
conftest.py/ cross-addonsharing — partial at best.
conftest.py(fixtures, cross-addon pathsharing) works. Largest change — effectively a second test-execution mode.
This needs a maintainer convention call (it touches every future addon and the
two core maintainers' differing styles), so it is intentionally not bundled
into #52 / PR 820.
References
test/+*_test.py+pytest addons)