feat(python): flows_to and the L3/L4 slice/flow verbs on the Python facade (#270, rc.1 leg) - #297
Open
rahlk wants to merge 15 commits into
Open
feat(python): flows_to and the L3/L4 slice/flow verbs on the Python facade (#270, rc.1 leg)#297rahlk wants to merge 15 commits into
rahlk wants to merge 15 commits into
Conversation
…ardening + pyfix goldens (#270)
…stead of crashing (#270)
…al provider (#270) _index() walked only mod.functions and one level of mod.types[*].callables, so closures (a callable's own nested callables) and nested classes (classes inside classes or inside callables) were invisible to program_graph/callable_of/resolve_location — silently degrading to seed-only with no note, even though the Neo4j emitter and the analyzer's own call-graph walker both recurse fully. _add/_add_type now recurse into getattr(..., "callables"/"types", None) defensively, so every callable that owns a body ends up indexed regardless of nesting depth, while staying a no-op on model families (e.g. the cldk-native cpg.models.Node) that don't support nested classes at all. Also fixes two vacuous assertions in the same test file surfaced alongside this change: a `... or True` tautology, and an `and`/`or` precedence bug that made a source-slice assertion pass regardless of its left operand.
…_slice contract (#270) resolve_location: the local mixin accepts a basename/suffix seed (path.endswith("/" + file)) in addition to an exact module-path match, and a golden test already pins this; the Neo4j backend only matched exactly, so the same seed string behaved differently per backend. Cypher now reads (n._module = $file OR n._module ENDS WITH $suffix). source_slice: aligned to the adjudicated three-way contract (vertex exists without a span/start_line -> (module_path, None); vertex unknown/not found -> (None, None); never fabricate a line number by parsing the vertex key's shape). The previous implementation degraded synthetic ports (@entry, @formal_in:N, ...) to (None, None) instead of (module_path, None), and worse, could fabricate a plausible-looking "module:line" for a vertex that doesn't exist at all whenever its key happened to parse as line:col shape. Rewritten to resolve the exact vertex by comparing _to_uri(n.id) against the requested vertex_uri, scoped to the owning callable's PyCFGNode rows (one round trip). sdg_edges: scoped the destination side too (b._module IN mods), matching the existing source-side scoping and every other query in this backend.
…n, flows_to, slice_forward (#270) program_graph parity now includes prov (as a tuple) in the edge-comparison key, so a backend that dropped or reordered ddg provenance no longer passes silently. Two new tests assert full-coverage parity rather than one hand-picked seed per verb: source_slice/callable_of agreement for every vertex of every callable, and resolve_location's full hit-list agreement for every (file, line) that owns a real vertex. test_verb_parity now also drives flows_to (fed a real param_in pair discovered dynamically from the live application, since the fixture's app_name is a random tmp-dir name each run) and slice_forward, alongside the existing slice_backward/ def_use/control_deps. Also corrects the module docstring's claim that the analyzer's bolt writer prunes other applications globally per emit -- it scopes the orphan-module prune to this application's own app_name; the dedicated-container advice is kept for run isolation, not because of any cross-application risk.
…comments (#270) max_level capture is now unconditional (self._max_level = analysis.max_level, no hasattr guard) and max_level() drops its getattr(..., 1) fallback -- an envelope that somehow lacks max_level now fails loudly instead of silently reporting level 1. _StubAnalysis in the schema-contract test gains a max_level default to match. Kept the hasattr(self, "_level_int") guard around opts["analysis_level"]: verified it IS load-bearing -- test_python_schema_contract.py's _bare_local_backend() exercises _run_analyzer on a __new__ instance that never set _level_int, to isolate the schema-envelope gate from analyzer construction. Added a comment naming that test as the reason instead of removing it. Dropped the unused json/Path imports from test_facade_delegates.py.
2 tasks
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Part of #270 (rc.1 leg of the staged rollout — rc.2 TS, rc.3 Java, rc.4 Go, rc.5 C++, 2.0.0 Rust engine follow).
What this adds
Five verbs on
PythonAnalysis, backed by the shared engine merged in #271, on both backends:slice_backward/slice_forward/control_deps/def_use/flows_to(interprocedural at L4, witness paths with per-hop kind/var/confidence)AnalysisLevelmaps to the analyzer's-a 1..4;max_levelis recorded from the same in-process run (never sniffed); underscore level names ("call_graph") accepted alongside enum valuesCpgLocalProviderMixin: language-neutral local provider over the (schema-2.0-shaped) upstream models — mintscan://<callable>@<key>vertex URIs matching the analyzers' ownparam_in/param_outvocabulary; recurses into closures and inner-class methodsPyNeo4jBackend: the five provider primitives in Cypher over thePY_*L3/L4 overlay, speaking the emitter's real can:// vertex ids (closes PyNeo4jBackend._to_uri appends a spurious trailing '@' to every vertex id #295 found mid-branch by the live parity suite)cldk.graphpublic exports; capability gate degrades honestly below level floors,strict=TrueraisesVerification
tests/graph+tests/analysis/python(exact-set goldens hand-derived from the committedpy-a4.jsonL4 sample)cldk/itself at-a 4: 8/8 legacy accessors, 6/6 new-verb calls, genuine cross-callableflows_towitness, honest degrade at L2Known deferrals (rc.2 touch-up candidates)
resolve_locationhas the Cypher fix but no discriminating stub test (stub ignores params; live suite only probes full paths)_MODULES_CTErecomputes app scoping per call instead of reusing cachedself._modules_index()(no public callable-enumerator yet)max_levelon thePyApplicationnode; SDK currently derives it from overlay presence)#270 stays open for the remaining legs.