fix: Do not load the native extension to build an empty rules list - #342
Conversation
|
Might be a parity gap, related to this area (not caused by this PR): rules added after construction are honored by Rust but not Python v0. Rust's |
Above comment is non-blocking, so I would approve this one and if needed we can discuss more here |
55653b2 to
9aea8fe
Compare
| # set. ``SymbolTable.expr_host_rules`` stores this untouched | ||
| # (``Optional[list[Any]]``) and only crosses into Rust at evaluation | ||
| # time, so seeding ``[]`` costs nothing here. | ||
| return [] |
There was a problem hiding this comment.
The empty-rules fast path changes behavior for one case: no rules + openjd.expr unavailable (pre-EXPR openjd-model, or the native extension not built for the platform).
- Before: the
try/except ImportErrorran even with no rules, so this case returnedNone. - After: it returns
[]unconditionally.
Two things worth confirming:
- The docstring is now stale — it still says the method "returns …
Nonewhen the engine bindings are unavailable (pre-EXPR openjd-model)", but with no rules that path is no longer reachable;Nonenow only happens with rules present + bindings missing. _expr_host_rulesis assigned straight ontosymtab.expr_host_rules(line 1621) and crosses into Rust at evaluation time. Within this repo nothing distinguishesNonefrom[](nois Nonechecks on it), so this looks safe — but please confirm openjd-model's SymbolTable / the Rust boundary treats an empty-list host-rules context the same as an unset/Noneone for a pre-EXPR build, so seeding[]there cannot trigger host-context construction thatNonepreviously avoided.
If both hold, a one-line docstring tweak so the None-return note reflects that it is now the rules-present case only would keep it accurate.
Session.__init__ calls _build_expr_host_rules() unconditionally, and that imported openjd.expr -- a facade over the native openjd._openjd_rs extension -- before checking whether there was anything to convert. With no path mapping rules it imported the engine, iterated an empty list, and returned []. So a worker that never evaluates an EXPR expression still loaded the extension on its FIRST SESSION: 49d1804 moved the load from import time to first-session time rather than eliminating it. Reported by a reviewer reading the call site; confirmed by measurement. Return [] directly when there are no rules. That is byte-identical to what the loop produced, and it must stay [] rather than None because the session is still host scope and apply_path_mapping() has to remain available with an empty rule set. Seeding [] is free: SymbolTable stores expr_host_rules untouched as Optional[list[Any]] and only crosses into Rust at evaluation time, in _expr_support.symtab_to_expr_values -- verified that assigning None, [], and running a full non-EXPR resolve all leave the extension unloaded. Audited every load path rather than grepping for imports, using a sys.meta_path finder that records the stack at the moment openjd._openjd_rs is first imported, one flow per fresh interpreter. Before: four flows leaked -- Session() with no rules, with rules=None, a non-EXPR run_task, and a non-EXPR enter_environment -- and all four traced to the same single frame, _session.py:476 __init__ -> _build_expr_host_rules. After: every non-EXPR flow is clean, and the only remaining loads are the three that should load. Import of openjd.sessions, non-EXPR argument resolution, and non-EXPR optional-int resolution were already clean and stayed clean. Known limitation, asserted rather than left implicit: a session with real path mapping rules still loads the extension, because the rules must be converted into openjd.expr.PathMappingRule engine objects to seed host context. openjd-sessions cannot avoid that alone; closing it needs openjd-model to accept unconverted rules and convert them at the Rust boundary it already crosses. test_path_mapping_rules_do_load_the_extension pins the current behaviour so that a future fix is noticed here instead of passing silently. This matters in practice: a worker with path mapping rules configured -- the common case -- still loads the extension. Correcting my own earlier audit: I classified this site as "already function-local (lazy)" and moved on. Lazy is not conditional. A function-local import still runs unconditionally the moment its function is called, and this one is called from __init__. The existing purity tests covered import time plus one runtime path, so nothing would have caught it. Mutation-checked, 4 mutants, 0 survivors: neutralising the fast path, returning None instead of [], firing it for every session, and inverting its condition are each caught by name. Verified: 913 passed / 39 skipped / 16 xfailed; ruff, black, mypy native and mypy --platform win32 all clean. Signed-off-by: David Leong <116610336+leongdl@users.noreply.github.com>
9aea8fe to
75863c5
Compare
|
|
||
| context = ModelParsingContext(supported_extensions=["FEATURE_BUNDLE_1"]) | ||
| script = StepScript.model_validate( | ||
| {"actions": {"onRun": {"command": "echo", "args": ["hello"]}}}, |
There was a problem hiding this comment.
run_task resolves the command via shutil.which (_win32/_locate_executable.py:66), and on Windows echo is a cmd.exe builtin with no standalone echo.exe on PATH — so shutil.which("echo") returns None and the launch raises RuntimeError("Could not find executable file: echo"). The probe then exits non-zero and _run_probe's assert completed.returncode == 0 fails.
The rest of this file goes out of its way to support Windows (see _probe_python), and the existing suite already handles exactly this: test_runner_base.py:835 uses "echo" if is_posix() else "cmd.exe". This end-to-end test uses echo unconditionally, so test_running_a_non_expr_task_stays_pure_end_to_end looks like it will fail on Windows. Consider selecting the command per-platform (e.g. cmd.exe /c echo hello).
The gap #341 left
#341stoppedimport openjd.sessionsfrom loading the nativeopenjd._openjd_rsextension. ButSession.__init__calls_build_expr_host_rules()unconditionally, and that importedopenjd.expr—a facade over the extension — before checking whether there was anything to
convert. With no path mapping rules it imported the engine, iterated an empty
list, and returned
[].So a worker that never evaluates an EXPR expression still loaded the extension on
its first session.
49d1804moved the load from import time tofirst-session time rather than eliminating it. Caught by a reviewer reading the
call site; confirmed by measurement:
The fix
Return
[]directly when there are no rules — byte-identical to what the loopproduced.
It must stay
[]and notNone: the session is still host scope, soapply_path_mapping()has to remain available with an empty rule set. Seeding[]is free —SymbolTablestoresexpr_host_rulesuntouched asOptional[list[Any]]and only crosses into Rust at evaluation time in_expr_support.symtab_to_expr_values. Verified that assigningNone, assigning[], and running a full non-EXPR resolve all leave the extension unloaded.Audited every load path, not just this one
Rather than grepping for imports, I installed a
sys.meta_pathfinder thatrecords the stack at the moment
openjd._openjd_rsis first imported, and raneach flow in its own fresh interpreter.
import openjd.sessionsSession()no rulesSession(path_mapping_rules=None)run_taskend to endenter_environmentSession(path_mapping_rules=[rule])openjd.sessions._v1All four leaks traced to the same single frame:
After the fix every non-EXPR flow is clean and the only remaining loads are the
three that should load.
Known limitation, asserted rather than implied
A session with real path mapping rules still loads the extension, because the
rules must be converted into
openjd.expr.PathMappingRuleengine objects to seedhost context. openjd-sessions cannot avoid that alone — closing it needs
openjd-model to accept unconverted rules and convert them at the Rust boundary it
already crosses.
test_path_mapping_rules_do_load_the_extensionpins the current behaviour so afuture fix is noticed here instead of passing silently.
This matters in practice: a worker with path mapping rules configured — the
common case — still loads the extension on its first session. This PR fixes the
no-rules case, which covers the CLI default and most tests, and narrows the
remaining exposure to one well-understood site. It does not deliver "a non-EXPR
worker never loads the extension" on its own.
Correcting my own earlier audit
When I audited crossings for #341 I classified this site as "already
function-local (lazy)" and moved on. Lazy is not conditional. A
function-local import still runs unconditionally the moment its function is
called, and this one is called from
__init__. The existing purity tests coveredimport time plus one runtime path, so nothing would have caught this.
Testing
Mutation-checked: 4 mutants, 0 survivors. Neutralising the fast path,
returning
Noneinstead of[], firing it for every session, and inverting itscondition are each caught by a named test.