Skip to content

Variant complex nodes, on by default (deltasignal specs/046) - #109

Merged
adamjohnwright merged 13 commits into
mainfrom
feat/variant-nodes
Oct 10, 2026
Merged

adamjohnwright merged 13 commits into
mainfrom
feat/variant-nodes

Conversation

@adamjohnwright

Copy link
Copy Markdown
Contributor

Variant complex nodes (deltasignal specs/046), on by default. LNG_VARIANT_NODES=0 restores the previous networks.

What it does

  • A complex that contains sets becomes one node per member combination. "CDK4,CDK6:CCND" (2 × 3 members) becomes 6 nodes with variant keys parent::variant::slot=member_... (src/variant_keys.py).
  • Each reaction is copied once per consistent choice across ALL its participants, catalysts and regulators included (decisions D2, D4). The copy that chose CDK4 + CCND1 produces exactly the CDK4:CCND1 node, so a downstream consumer joins it by key (src/variant_emission.py).
  • An output-only set is bound to the input set whose members correspond by isoform-level reference identity, so it is not multiplied.
  • Past LNG_MAX_VARIANTS (decision D6):
    1. pool the participants whose choice reaches no output (<stId>::pool, fed by variant_pool OR edges, which the solver combines by mean, decision D5);
    2. else one copy per output variant;
    3. else one copy whose plain outputs feed their variants (variant_split), restricted to unproduced nodes consumed by a curated following reaction.
  • Depleters: several variants of one curated catalyst deplete a shared target once, through a pool.
  • Regulators: a catalyst or regulator reads the copy of its entity that a curated preceding reaction produces.
  • Every export (nodes.csv, node_resolution, containment, drugs, boundary layer) understands variant keys and pool ids. A pool id is never looked up in Neo4j and never resolves as a readout or gene target.

Measured

Arm vn7, build 20261009-2042_1daf333_vn7, against canonical 20261004-1959_6990015, same solver:

net McNemar p
curator held-out +130 (233/103) 1e-12
curator tuning −18 0.43
experimental −10 (22/32) 0.22
  • Coverage is identical: 23,511 / 845 valid cases, 864 / 122 perturbations.
  • Cut hand-offs go 2,504 → 0; required inputs beyond the curated ones 13,048 → 0; missed joins 0.
  • The full record, including two failed arms (vn4, vn5) and every mechanism traced to a line, is in deltasignal specs/046-variant-complex-nodes/research.md.

Review

An adversarial review of the diff confirmed flag-off output is byte-identical to main on 4 regenerated pathways. It found 4 real defects, all fixed in 1daf333, each with a test that fails without it. One design caveat is recorded, not changed: a pool equals expansion exactly only for a single varying participant under a pure product.

Tests

  • 388 pass offline. The 2 failures are environmental: no output/ dir, and Neo4j unreachable by design.
  • The boundary-layer unit tests pin LNG_VARIANT_NODES=0, the path they were written for. The variant path is covered by tests/test_variant_keys.py, test_variant_emission.py and test_variant_pools.py.

🤖 Generated with Claude Code

adamjohnwright and others added 13 commits October 9, 2026 09:55
… per set per reaction), parser and leaves (deltasignal specs/046, decisions D1-D6)

Pure module, not yet wired into emission. 10 database-free tests.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
…er-reading sites through it (both id formats)

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
…ariant containment, variant-aware boundary layer

src/variant_emission.py builds reaction copies from variant keys: one choice
per set per reaction over inputs, outputs, catalysts and regulators (D2, D4);
output-only slots bound by isoform-level reference identity, ties by sorted
rank and counted (replaces F9's positional tie-break); over the cap, one copy
with plain stable ids (first step of D6). Phases 1-3 unchanged (D1).
containment.csv gives a variant only its chosen members and leaves; the
boundary layer decomposes a root variant through its chosen members and gives
a terminal variant sinks for its chosen leaves.

FGFR2 (R-HSA-5654738): required inputs beyond the curated ones 1,156 -> 0,
cut hand-offs 544 -> 0; all 1,782 variant input edges are fed, 0 missed joins
(I3b). Flag off: 360 tests pass, default output unchanged.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
…s (gene resolution); emitter unit tests; determinism verified (FGFR2 identical under PYTHONHASHSEED 0 and 7)

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
…(recursive fan-out); deep boundary components use variant-aware leaves

The first catalog build failed 6 of 92 pathways: 4 on a bound output set whose
mapped member complex holds a set of its own (IL-3 R-HSA-879909), 2 on a
boundary component past nesting depth 6 passed to get_terminal_components.
All 6 now build. Regression test added.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
…ectly from Neo4j; zero copies is an error

Coverage failure of the first catalog arm, traced: (1) the emitter was given
the reduced connectivity table, missing reactions with no preceding/following
event; it now receives the pathway's full reaction list from the generator;
(2) with the connector's entity caches primed for the pathway, a set outside
the prefetch reported no members, so its reaction got zero copies and was
dropped silently (PIP3: 8 of 89, among them the RTK catalyst set). The key
module now reads structure directly from Neo4j with its own memo, an empty
curated set is a leaf, and a reaction with zero copies raises.
PIP3: 89/89 reactions, 1,176 copies; ERBB2 forms in 84 nodes (were 0).

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
…d connector caches)

Under variant nodes, catalyst and regulator members bring in entities the
pathway's cache prefetch never covered; the primed caches reported them as
unlabelled leaves, so a member complex such as KIT's R-HSA-205310 became a
leaf and KIT had no root to perturb (24 experimental cases, KIT and PDGFRB in
PIP3). Under LNG_VARIANT_NODES the boundary layer's complex/set/component/leaf
tests go through src/variant_keys.py. Default path unchanged (367 tests pass
offline).

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
…regulators read the produced copy (specs/046 vn4 traces)

Three mechanisms traced in the vn4 arm (held-out -152, experimental -56):
- a capped reaction read plain ids nothing produced (IFN alpha/beta);
- each catalyst variant depleted a shared substrate, multiplying (TP53);
- a regulator took the first registry node of its entity, an unfed copy (PTEN).

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
…never looked up in Neo4j, never a readout or gene target

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
…s are its members; missing ones become roots); capped plain outputs feed their variants (variant_split)

vn5 traces: ROBO RPL10/RPL22/RPL5 resolved to no node (144 curator cases);
RAF R-HSA-5672980 and WNT R-HSA-3965447 wrote plain outputs nothing read
(experimental -34 and -8).

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
- depleters are pooled by the curated catalyst they came from (a bare set's
  members share no id prefix: 6 DUSPs each depleted the RAF MAPK3 dimer);
- an over-cap pool of a bare set reads its members' nodes (RAS GEFs, 645
  variants, was a pool with no input), else one root per member;
- step 1 of the cap fallback pools only participants sharing no slot with a
  kept one (decision D2);
- variant_split feeds only unproduced variant nodes consumed by a curated
  following reaction (decision D1; it diluted a 33-producer node);
- the boundary-layer docstring is a docstring again.

Each fix has a test that fails without it.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Measured on arm vn7 against the canonical catalog: curator held-out +130
(p 1e-12), tuning -18 (p 0.43), experimental -10 (p 0.22); coverage identical;
cut hand-offs 2,504 -> 0. LNG_VARIANT_NODES=0 restores the previous networks.

The boundary-layer unit tests pin the flag-off path they were written for
(mocked connector lookups); the variant path is covered by test_variant_*.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
@adamjohnwright
adamjohnwright merged commit 132d4b8 into main Oct 10, 2026
4 checks passed
@adamjohnwright
adamjohnwright deleted the feat/variant-nodes branch October 10, 2026 03:00
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant