Prefer an installed plugin package over a same-named folder in the working directory - #2419
Open
Flix6x wants to merge 4 commits into
Open
Prefer an installed plugin package over a same-named folder in the working directory#2419Flix6x wants to merge 4 commits into
Flix6x wants to merge 4 commits into
Conversation
…med folder Context: - GH issue #2415: os.path.exists on a bare FLEXMEASURES_PLUGINS entry is a check relative to the working directory, so starting FlexMeasures from a plugin's own repository made the loader take the file path branch for an installed plugin. That re-executes __init__.py under a new module object while submodules keep referring to the old one, so the Blueprint that gets registered is the empty one and the plugin's routes and CLI commands go missing, silently Change: - Look the name up with importlib.util.find_spec first, and import the package when one is found, unless the entry is spelled out as a path (absolute or containing a separator), which still loads exactly the folder it points to - Ignore namespace packages when looking up, so a folder without an __init__.py keeps reporting that, rather than loading as an empty package - Warn when a bare name resolves to a folder in the working directory, as that is the case the loader cannot distinguish from a typo'd package name Signed-off-by: F.N. Claessen <felix@seita.nl>
Context: - The loader had no tests, and the bug in #2415 was invisible in the loaded-plugins listing: only the routing table shows that the registered Blueprint lost its routes Change: - Added test_plugin_utils.py, writing throwaway plugins whose Blueprint gets its route from a submodule, as real plugins do, and asserting on the resulting url_map - Covers the shadowing case, a bare name with nothing installed, relative and absolute path entries, a folder without __init__.py, and a name that is neither Signed-off-by: F.N. Claessen <felix@seita.nl>
Context: - The precedence between an installed package and a same-named folder was neither documented nor obvious Change: - Documented it under FLEXMEASURES_PLUGINS, including how to load a folder on purpose - Added a changelog entry Signed-off-by: F.N. Claessen <felix@seita.nl>
Context: - The entry was written before the PR existed Change: - Pointed it at PR #2419 Signed-off-by: F.N. Claessen <felix@seita.nl>
Documentation build overview
5 files changed ·
|
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
The bug
register_pluginsdecided whether aFLEXMEASURES_PLUGINSentry is a file path or an installed package withos.path.exists(plugin). On a bare name likemy_plugin, that is a check relative to the working directory. So an installed plugin was loaded by path whenever a folder of that name happened to sit in the cwd — which is exactly the case when you start FlexMeasures from the plugin's own repository.Loading by path re-executes
__init__.pyunder a new module object and replacessys.modules[plugin_name]. With the common Blueprint layout (__init__.pycreates the Blueprint,views.pyimports it and attaches routes), the submodules imported by the first execution keep referring to the old module, so the Blueprint that FlexMeasures registers is a fresh, empty one. The plugin shows up as loaded, and its routes 404 and its CLI group is empty.Closes #2415.
The fix
As @nhoening put it in the issue: "If the package is installed, that should be used."
The name is now looked up with
importlib.util.find_specfirst, and imported as a package when one is found. An entry that is spelled out as a path (absolute, or containing a separator, e.g../my_plugin) still loads exactly the folder it points to, so the documented file-path usage is unchanged.Two details worth a look:
__init__.pyis importable as a namespace package, so accepting it would have replaced today's clear "does not contain an__init__.pyfile" error with an empty package and a vaguer "no blueprints found" warning.Does this change our current preference?
Yes, deliberately, and only for the ambiguous case: a bare name for which both an importable package and a same-named folder exist. That used to load the folder, and now loads the package. Everything else resolves as before:
my_pluginmy_pluginmy_plugin./my_pluginor/abs/my_pluginThe test
flexmeasures/utils/tests/test_plugin_utils.pyis new — the loader had no tests. Each test writes a throwaway plugin whose Blueprint gets its route from a submodule, the way real plugins are laid out, and asserts on the app'surl_map. That matters:LOADED_PLUGINSlooked perfectly healthy while the bug was live, and only the routing table shows that the registered Blueprint lost its routes.Covered: the shadowing case (regression), a bare name with nothing installed, relative and absolute path entries, a folder without
__init__.py, and a name that is neither installed nor present.Mutation-tested (each mutant applied to
plugin_utils.py, tests rerun):prefer_package = False)is_written_as_pathalwaysFalse(paths treated as names)is_written_as_pathalwaysTrue(names treated as paths)Pristine: 6 passed; full
flexmeasures/utilspackage: 253 passed.How to test
Reproduce as in the issue:
pip install -e .a plugin whose__init__.pydefines the Blueprint and whoseviews.pyadds a route, setFLEXMEASURES_PLUGINS = ["my_plugin"], and start FlexMeasures from the plugin repo's root. The route used to 404; it now works, and the loaded module is the installed one.🤖 Generated with Claude Code
https://claude.ai/code/session_01FxXYu3pPcB23HaUie3aK2d