Skip to content

CI test convention: support pytest / test/ / *_test.py, or require addons to conform #53

Description

@eduralph

Summary

The CI test runner supports exactly one test convention: a tests/ (plural)
directory, test_*.py (prefix) filenames, and Python's unittest loader.
Issue #52 made discovery recursive and added a per-addon import root so
nested-package addons that follow this convention (e.g. NameSuite, upstream
PR 941 — 20 modules / 234 tests now run) work. But a second in-flight
submission uses a different convention, which #52 deliberately did not
broaden to (narrow-scope decision). This issue records that gap so we can decide
it on its own.

The divergence (dsblank PR 1)

dsblank/addons-source PR 1 ("Add FastFiltersLib, SQLiteFastFilters,
PostgreSQLFastFilters") ships its tests as:

  • test/ (singular) directory — not tests/.
  • *_test.py (suffix) filenames — e.g. filter_overrides_test.py,
    sql_compat_test.py — not test_*.py.
  • pytest, via conftest.py that does the sys.path setup, including
    cross-addon sharing (SQLite/PostgreSQL conftest.py add FastFiltersLib/
    and FastFiltersLib/test/ to the path to reuse shared base classes), and a
    sibling import from filter_overrides_base import ….

Consequences for our runner:

  • Discovery (*/tests/**/test_*.py) does not match test/ + *_test.py,
    so these suites are never found.
  • Even if found, the unittest loader does not execute conftest.py, so
    the sys.path setup (and the cross-addon sharing) would not run — from fastfilterslib… import / from filter_overrides_base import would fail.

Decision needed

Pick one (or a mix):

  1. Require addons to conform to the documented convention (tests/ +
    test_*.py + unittest, with per-addon import root and relative imports for
    private helpers). Lowest CI cost; pushes the work to each addon.
  2. Broaden discovery to also accept test/ and *_test.py (still unittest).
    Cheap glob change, but does not solve pytest conftest.py / cross-addon
    sharing — partial at best.
  3. Add pytest as a CI runner so conftest.py (fixtures, cross-addon path
    sharing) works. Largest change — effectively a second test-execution mode.

This needs a maintainer convention call (it touches every future addon and the
two core maintainers' differing styles), so it is intentionally not bundled
into #52 / PR 820.

References

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions