Skip to content

20 plugin tests hardcode a developer's home path and override --core, so tests can run against the wrong checkout #467

Description

@ChuckBuilds

Summary

Twenty test files across sixteen plugins hardcode an absolute path to one developer's checkout, then fall back to guessing a sibling layout. Both are inserted at sys.path[0], so they override the core that scripts/run_plugin_tests.py --core and LEDMATRIX_CORE were told to use.

The practical effect is that you cannot point the plugin tests at a specific core. Ten tests in my run resolved src.common from a completely different checkout than the one I passed on the command line.

The code

plugins/baseball-scoreboard/test_idle_league_backoff.py:46-52, repeated verbatim in 19 other files:

plugin_dir = Path(__file__).resolve().parent
sys.path.insert(0, str(plugin_dir))
for candidate in (Path("/home/rackpi/projects/LEDMatrix"),
                  plugin_dir.parents[2] / "LEDMatrix"):
    if (candidate / "src" / "plugin_system" / "base_plugin.py").exists():
        sys.path.insert(0, str(candidate))
        break

Three problems, in increasing order of consequence:

  1. /home/rackpi/projects/LEDMatrix is a specific machine's home directory, committed to a public repo. It resolves for exactly one person and leaks a username.
  2. The fallback guesses a layout. plugin_dir.parents[2] / "LEDMatrix" assumes the plugins tree sits beside a directory literally named LEDMatrix.
  3. Both use sys.path.insert(0, ...), which beats the PYTHONPATH the runner sets from --core. So the flag that exists to choose a core is silently ignored whenever either candidate happens to exist.

Reproduction

Two core checkouts on one machine — /home/hdpi/LEDMatrix (older branch) and /home/hdpi/core-main (current main) — and a plugins tree at /home/hdpi/plugins-fix:

$ cd /home/hdpi/plugins-fix
$ python3 scripts/run_plugin_tests.py --all --core /home/hdpi/core-main
...
  - baseball-scoreboard/test_idle_league_backoff.py: exit 1 |
      ModuleNotFoundError: No module named 'src.common.sports_shared'
  - football-scoreboard/test_idle_league_backoff.py: exit 1 |
      ImportError: cannot import name 'sports_card' from 'src.common'
      (/home/hdpi/LEDMatrix/src/common/__init__.py)

That parenthesised path is the giveaway: --core /home/hdpi/core-main was passed, and the import came from /home/hdpi/LEDMatrix. parents[2] resolved to /home/hdpi, /home/hdpi/LEDMatrix exists, and it went in ahead of everything.

src.common.sports_shared exists in the core I asked for and not in the one it used, so all ten failures are artefacts of picking the wrong core.

Affected files

20 files in: afl-scoreboard, baseball-scoreboard, basketball-scoreboard, f1-scoreboard, football-scoreboard, geochron, hockey-scoreboard, lacrosse-scoreboard, ledmatrix-flights, ledmatrix-stocks, ledmatrix-weather, news, nrl-scoreboard, odds-ticker, soccer-scoreboard, ufc-scoreboard.

grep -rln "/home/rackpi" plugins/ --include=*.py

Suggested fix

Honour the contract the runner already provides. run_plugin_tests.py sets LEDMATRIX_CORE to an absolute path precisely so a child does not have to guess:

plugin_dir = Path(__file__).resolve().parent
sys.path.insert(0, str(plugin_dir))
core = os.environ.get("LEDMATRIX_CORE")
if core:
    sys.path.insert(0, core)
else:
    # only guess when nobody told us
    for candidate in (plugin_dir.parents[2] / "LEDMatrix", ...):
        ...

The hardcoded /home/rackpi/... entry should go regardless of the rest.

Given it is copied into twenty files, a shared helper — or a conftest.py at the plugins root — would stop the next copy drifting.

Why this matters beyond tidiness

The runner's whole purpose is running plugin tests against a nominated core, which is what CI does (--core "$GITHUB_WORKSPACE/core"). In CI the guess and the flag currently agree, so it passes — but the flag is not what is being honoured, and any layout where they disagree tests the wrong thing while reporting success or failure with equal confidence.

Environment

Found while validating ChuckBuilds/LEDMatrix#534 and #466 on a 256x64 rig: 238 passed, 1 skipped, 10 failed, and all ten failures were this.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions