From cdf1ec6cdab2e771ba8f71ebbe04590c2aadd7cb Mon Sep 17 00:00:00 2001 From: Adam Wright <668955+adamjohnwright@users.noreply.github.com> Date: Sat, 10 Oct 2026 02:31:05 -0400 Subject: [PATCH 01/10] specs/048: pre-register two experimental levers (pathogen nodes inert; cycles closed through a pooled set) Re-traced on canonical 20261009-2307_132d4b8: PDPK1 KO still reads 100x through the SARS-CoV-2 N:M:PDPK1 inhibitor (7 cases); the RAF scaffold loop of specs/047 trace 3 is gone, replaced by the MAPK <-> p-MAPK cycle closing through R-HSA-169291::pool, which pool detection cannot see (12 of 21 RAF failures). Co-Authored-By: Claude Opus 5.5 (1M context) --- specs/048-experimental-levers/spec.md | 153 ++++++++++++++++++++++++++ 1 file changed, 153 insertions(+) create mode 100644 specs/048-experimental-levers/spec.md diff --git a/specs/048-experimental-levers/spec.md b/specs/048-experimental-levers/spec.md new file mode 100644 index 0000000..384f25f --- /dev/null +++ b/specs/048-experimental-levers/spec.md @@ -0,0 +1,153 @@ +# specs/048: two experimental levers from specs/047 (pre-registration) + +**Goal.** Close part of the experimental gap to MP-BioPath (68.79% against +75.74%, canonical build `20261009-2307_132d4b8`, solver `0f0a11e`). +specs/047 showed that the gap comes from how the networks are built, not from +the solver. This spec takes the two narrowest traced classes, re-traced on the +new canonical build, and pre-registers one arm for each. **Experimental is the +primary axis.** + +Both arms are scored against the canonical build at solver main, code +defaults, with the `root_cycle` pin protocol. + +## Lever A: pathogen-derived nodes held at baseline + +**Re-traced on the canonical build.** The specs/047 trace 2 still holds. +- PDPK1 knockout reads 100× at all 7 DOWN readouts in PIP3 (199844, 198335, + 9614997, 200163, 202072, 202074, 198373); truth is DOWN. +- "PDPK1 phosphorylates AKT at T308" (R-HSA-198270) has a negative regulator, + `N:M:PDPK1` (R-HSA-9755778). That is the SARS-CoV-2 N dimer and M protein + bound to PDPK1. +- A PDPK1 knockout zeroes the complex and de-represses the reaction, and the + AKT loop rails. +- In a cell without the virus, the complex does not exist. + +**Census** (`~/deltasignal-catalogs/analysis/048/pathogen_census.py`, Neo4j +release 97). The census runs over every entity in the canonical catalog. +- **Rule:** a leaf is pathogen-derived when it has a species and none is Homo + sapiens; a complex is, when any component is; a set is, when every member + is. This is the specs/032 drug rule, with species in place of the `Drug` + class. +- **Count:** 65 of 16,359 entities, 20 of them leaves, in 9 pathways. +- **Pathways:** + - PIP3 (R-HSA-1257604), VEGF (194138) and CD28 (388841): SARS-CoV-2 N and M. + - DDX58/IFIH1 (168928): 45 entities, viral RNAs and proteins. + - IFN α/β (909733), Class I MHC (983169), NK-cell receptors (198933), + death receptors (140534), and the E-protein complexes in 446728. +- **Experimental reach:** of the ten experimental pathways, only PIP3 is + touched. + +**Design.** +- **Generator:** writes `pathogens.csv` beside `drugs.csv`, by the rule above, + with variant ids resolved as `drugs.csv` resolves them. The file is always + written, even when it lists nothing, and the networks are otherwise + unchanged. +- **Solver:** `DS_PATHOGEN_MODE=propagate` (default) or `inert`. Under `inert`, + listed nodes are pinned at baseline, as drugs are under `DS_DRUG_MODE=inert`: + fold 1, never 0, and an explicit observation wins. The solve reports + `pathogen_rule` and `pathogens_held`. The bench refuses an `inert` arm whose + build has no `pathogens.csv`. + +**Predictions** (arm `path`, against the same build under `propagate`): +- **A-P1.** At least 5 of the 7 PDPK1-knockout DOWN cases become correct. +- **A-P2.** The 3 PDPK1-overexpression UP cases stay correct. +- **A-P3.** Experimental net is +3 to +7, all in PIP3. The other nine + experimental pathways move by 0. +- **A-P4.** Curator held-out net is within ±15. No direction is predicted: in + DDX58/IFIH1 the viral RNAs are the pathway's own triggers, so holding them + at baseline is faithful to an uninfected cell, but that may not be what + the 2019 curator ground truth assumes. + +## Lever B: modification cycles that close through a pooled set + +**Re-traced on the canonical build, which replaces specs/047 trace 3.** +- The scaffold loop of trace 3 is gone. Under variant nodes, the scaffold + released by R-HSA-5672980 is a dead end, and the binding step reads + `R-HSA-5672717::pool` instead. +- RAF is now 28/49, against MP-BioPath's 45 and 39 on the previous canonical + build. +- 12 of the 21 failures are one class. ARAF, BRAF, RAF1 or NRAS knockout reads + p-T,Y MAPK readouts (1268261, 5674340, 5674341) at 36–70×; truth is DOWN. + +**Trace: NRAS KO → R-HSA-1268261 = 53.8×** +(`~/deltasignal-catalogs/analysis/048/trace_up.jl`, output in `nras_up.txt`). +The rising path is a curated modification cycle of MAPK: +1. **R-HSA-5672972:** MAPKs (R-HSA-169291) bind the activated RAF complex. +2. **R-HSA-5672978:** RAF phosphorylates MAP2K. +3. **R-HSA-5672973:** MAP2K phosphorylates MAPK. +4. **R-HSA-5672980:** dissociation releases p-T,Y MAPKs (R-HSA-169289). +5. **R-HSA-5675376:** DUSPs dephosphorylate them, giving MAPK monomers and + dimers (R-HSA-5675361). +6. The released MAPK3 and MAPK1 nodes feed `R-HSA-169291::pool` (over the + variant cap), which re-enters step 1. The pool node reads 53.8×. + +This is MAPK ⇄ p-MAPK, the shape specs/039's balance pools solve. The RAF +build reports 4 pools, all RAS GDP/GTP, and **0 dropped candidates**: MAPK was +never a candidate. There are two reasons, both in `find_pools`: +- **B1. Membership hop.** The R-graph is built from reaction steps only. The + return from the MAPK3 node into `169291::pool` is a `variant_pool` + membership edge, not a reaction, so for pool detection the cycle is open, + while for the solver it is closed. This is a specs/046 seam. +- **B2. Regeneration by entity id.** Amendment 5 keeps a multi-step path only + if every non-R, non-small input it consumes is output again, compared by + stable id. Step 1 consumes R-HSA-5672718 (RAS:GTP:activated RAF dimers) and + R-HSA-5672716 (MAP2K dimers). Step 4 releases R-HSA-5672712 (RAS:GTP: + activated RAF dimers, a different stable id with the same proteins) and + R-HSA-5672721 (p-2S MAP2K dimers, the same proteins, modified). Nothing is + lost, but the ids differ, so the path would be dropped. + +**Design** (generator only; the solver is unchanged): +- **B1:** `LNG_POOL_JOIN_MEMBERS=1`. When a step copy's R-containing input is + a pool node (`variant_pool` or `set_member` target), the step is taken from + each member node of that pool that contains R. The transition row's + `source_uuid` is the member. The solver already re-evaluates pool-fed + inputs of a pool's steps with the pool at baseline (specs/039 amendment + 5), so the pool node is read correctly in the drive. +- **B2:** `LNG_POOL_REGEN=protein`. A consumed partner counts as regenerated + when every protein it carries is output again by a step of the same path, + in any form. `entity` (default) keeps amendment 5. Small molecules and + ubiquitin stay exempt. +- **One arm with both flags.** B1 alone recovers nothing in RAF, because B2 + then drops the path. Other pathways gain pools too; they are counted + before scoring. + +**Structural checks** (must hold before any score is read): +1. The build completes for all 92 pathways. +2. RAF has a pool whose states include a MAPK3 node and a p-MAPK3 node, and + likewise for MAPK1. +3. Coverage (valid cases, `Pinned:` count) is within 1% of the control on + both axes. +4. The new pools are counted per pathway (pools, states, multi-step paths) + and recorded before the score is read. + +**Predictions** (arm `mpool`): +- **B-P1.** At least 6 of the 12 RAF knockout→UP cases become correct (DOWN). +- **B-P2.** RAF experimental rises to ≥ 34/49. +- **B-P3.** Experimental net is ≥ +5. Changes outside RAF are not predicted + in direction: ERBB2, PIP3 and WNT may gain pools. +- **B-P4.** No curator held-out direction is predicted. Pools change wherever + MAP kinase cycles or set-closed cycles exist (MAPK cascades appear in many + signalling pathways). + +## Decision rule (both arms, separately) + +- **Adopt** if: + - experimental net > 0; + - curator held-out net ≥ −15; + - no single pathway loses more than 10 cases on either axis; + - the structural checks hold. +- Any loss beyond a floor is traced to a case before anything else. +- **Report:** both axes, held-out and tuning, McNemar p, the pathways and + distinct readouts moved, convergence, and each prediction marked met or + missed. +- **The arms are independent.** Each is decided against the control on its + own. If both are adopted, the combination is measured once before it + becomes canonical. + +## Not in scope + +These specs/047 classes are left for later specs: +- the PTEN mRNA silo (trace 1); +- the Mitotic G1 E2F release silo (trace 4); +- TP53 DAXX over-coupling; +- THEM4, 7 PIP3 failures, not yet traced. From b9980582f180980078db0ae941aae579d44bb061 Mon Sep 17 00:00:00 2001 From: Adam Wright <668955+adamjohnwright@users.noreply.github.com> Date: Sat, 10 Oct 2026 03:01:46 -0400 Subject: [PATCH 02/10] specs/048 amendment 1: lever B re-scoped before any arm (homodimer binding bug B3; shared-intermediate pools unresolved); arms path/pathctl and bind registered Co-Authored-By: Claude Opus 5.5 (1M context) --- specs/048-experimental-levers/spec.md | 68 +++++++++++++++++++++++++++ 1 file changed, 68 insertions(+) diff --git a/specs/048-experimental-levers/spec.md b/specs/048-experimental-levers/spec.md index 384f25f..640ca22 100644 --- a/specs/048-experimental-levers/spec.md +++ b/specs/048-experimental-levers/spec.md @@ -151,3 +151,71 @@ These specs/047 classes are left for later specs: - the Mitotic G1 E2F release silo (trace 4); - TP53 DAXX over-coupling; - THEM4, 7 PIP3 failures, not yet traced. + +## Amendment 1 (2026-10-10, before any arm ran): lever B re-scoped + +I implemented B1 and B2 and generated RAF alone with them +(`~/deltasignal-catalogs/analysis/048/raf_probe`). It found two more breaks +in the MAPK cycle. + +**B1 needs a second hop.** "Dissociation of RAS:RAF complex" (R-HSA-5672980) +writes the plain node R-HSA-169289 (p-T,Y MAPKs). That node reaches p-MAPK1 +and p-MAPK3 by `variant_split` edges, not by a reaction. With that hop +added, the release step ends at p-MAPK1. + +**B3: a binding bug in the generator, not a pool question.** +- "Cytosolic DUSPs dephosphorylate MAPKs" (R-HSA-5675376) is curated as + monomers → monomers and dimers → dimers. Our copies bind them crosswise: + - p-MAPK1 (R-HSA-109853) → MAPK1 dimer (R-HSA-5675354), a dead end; + - p-MAPK1 dimer (R-HSA-109855) → MAPK1 (R-HSA-59282). +- **Cause.** `_reference_signature` says "a member complex keeps its leaf + multiset". But it builds the signature from `get_terminal_components`, + which returns a set, so a homodimer gets its monomer's signature. The tie is + then broken by sorted id, and here that crosses the pairs. +- This is a specs/046 defect, and it applies wherever a set holds both a + protein and its homodimer. +- **Fix:** `LNG_BIND_STOICH=1`, which multiplies leaves by component + stoichiometry. Test: `test_homodimer_binds_to_homodimer_not_monomer`. With + the fix, the DUSP step returns p-MAPK1 to MAPK1, and pool detection finds + MAPK1 ⇄ p-MAPK1 and MAPK3 ⇄ p-MAPK3. + +**What still blocks lever B: pools that share intermediates.** +- The two MAPK pools share 246 nodes. The RAF:scaffold:MAP2K:MAPK complexes + are far over the variant cap, so MAPK1 and MAPK3 pass through the same + plain complex nodes. +- `find_pools` removes a node claimed by two pools, so both pools are dropped. +- Solving pools that share intermediates is a design question the spec did + not register: it needs a solver change, and it may warrant blind + derivations. +- B1 and B2 are kept on the generator branch `wip/048-mapk-pools`. **The + `mpool` arm does not run.** + +**Arms now registered:** +1. **`path`** (lever A), unchanged. + - Build: `--variant path` from the generator branch, with flags at their + defaults. Its networks are the canonical networks plus `pathogens.csv`. + - Two arms on that build: `pathctl` (`DS_PATHOGEN_MODE=propagate`) and + `path` (`inert`). + - `pathctl` must reproduce the canonical predictions exactly. If it does + not, the build differs, and the difference is traced before `path` is + read. +2. **`bind`** (B3 alone, a faithfulness fix). + - Build: `--variant bind --env LNG_BIND_STOICH=1`, against the canonical + build. + - **Structural checks:** + 1. The build is complete. + 2. Coverage is within 1% on both axes. + 3. On RAF, R-HSA-5675376 binds p-MAPK1 to MAPK1 and the p-MAPK1 dimer to + the MAPK1 dimer. + 4. The number of reactions whose bindings change is counted catalog-wide + before scoring. + - **Predictions:** + - **bind-P1.** No direction is predicted for either axis. The fix closes + curated cycles (MAPK ⇄ p-MAPK) that the solver will iterate, because + no pool covers them. The 046 record shows that closing cycles without + a pool has gone both ways. + - **bind-P2.** RAF moves. Its 12 knockout→UP cases are the readouts of + this cycle; the direction is not predicted. + - **Decision rule:** adopt on faithfulness if held-out ≥ −15 and + experimental ≥ −15 (the specs/046 rule for a representation fix). A + loss beyond either floor is traced first. From 4cb152d527997630cc1c696bbeaf4c1c9c31abc7 Mon Sep 17 00:00:00 2001 From: Adam Wright <668955+adamjohnwright@users.noreply.github.com> Date: Sat, 10 Oct 2026 03:12:13 -0400 Subject: [PATCH 03/10] DS_PATHOGEN_MODE (specs/048): pathogen-derived nodes held at baseline, as drugs are Reads the generator's pathogens.csv into ReactionNetwork.pathogen_stids (default-nothing 8th field, so existing constructors are unchanged), carries it through the CLI and API parse -> solve round trip, and pins the listed nodes at baseline under DS_PATHOGEN_MODE=inert with the drug rule's observation precedence. The solve reports pathogen_rule / pathogens_held; the bench refuses an inert arm on a build without the table or that held nothing, and run_arm.sh records the Pathogens line. Default propagate is byte-identical (asserted). test_drugs.jl 59 -> 121. Co-Authored-By: Claude Opus 5.5 (1M context) --- CLAUDE.md | 2 +- bench/benchmark_vs_mpbiopath.py | 12 +++- cli/deltasignal.jl | 6 +- scripts/run_arm.sh | 1 + src/DeltaSignal.jl | 1 + src/api/server.jl | 12 +++- src/core/drugs.jl | 43 ++++++++++++ src/io/tsv_parser.jl | 41 ++++++++--- src/solvers/steady_state.jl | 20 +++++- test/test_drugs.jl | 116 ++++++++++++++++++++++++++++++++ 10 files changed, 240 insertions(+), 14 deletions(-) diff --git a/CLAUDE.md b/CLAUDE.md index 7786a96..a2bb57b 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -538,7 +538,7 @@ other seven execute code and print output. | `test/test_and_curves.jl` | 71 | | | `test/test_scc_break_roles.jl` | 56 | | | `test/test_self_inhibition.jl` | 82 | specs/022, added 2026-09-25 | -| `test/test_drugs.jl` | 59 | specs/032, added 2026-09-26 | +| `test/test_drugs.jl` | 121 | specs/032, added 2026-09-26; pathogens (specs/048) 2026-10-10 | | `test/test_set_pool.jl` | 79 | specs/033, added 2026-09-26; `variant_pool` (specs/046) 2026-10-09 | | `test/test_conserved.jl` | 15 | specs/035, added 2026-09-26 | | `test/test_cycles.jl` | 236 | specs/039, added 2026-09-27 | diff --git a/bench/benchmark_vs_mpbiopath.py b/bench/benchmark_vs_mpbiopath.py index f251b27..f922ef6 100644 --- a/bench/benchmark_vs_mpbiopath.py +++ b/bench/benchmark_vs_mpbiopath.py @@ -1023,7 +1023,8 @@ def run_pathway(pathway_id: str, pathway_name: str, gene_to_stids_cache=None, "cofactor_stids": parsed.get("cofactor_stids"), # specs/032: without it an edge-drop arm under DS_DRUG_MODE=inert # reached the solver with no drug table (review of the branch). - "drug_stids": parsed.get("drug_stids")} + "drug_stids": parsed.get("drug_stids"), + "pathogen_stids": parsed.get("pathogen_stids")} # Use the server-cached network by id (fast: send only observations per # solve). But if SKIP_EDGE_TYPES modified the edges above, the cached # network is stale, so send the full modified payload instead. @@ -1065,11 +1066,16 @@ def run_pathway(pathway_id: str, pathway_name: str, gene_to_stids_cache=None, if ds_result and ds_result.get("drug_rule") == "inert: no drug table": raise SystemExit(f"{pathway_name}: DS_DRUG_MODE=inert but the network has no drugs.csv " "(a build that predates it), so no drug is held (specs/032).") + if ds_result and ds_result.get("pathogen_rule") == "inert: no pathogen table": + raise SystemExit(f"{pathway_name}: DS_PATHOGEN_MODE=inert but the network has no pathogens.csv " + "(a build that predates it), so nothing is held (specs/048).") if ds_result: SOLVE_TALLY["solves"] += 1 SOLVE_TALLY["converged"] += bool(ds_result.get("converged")) SOLVE_TALLY[f"drug_rule={ds_result.get('drug_rule', 'unknown')}"] += 1 SOLVE_TALLY["drugs_held"] += int(ds_result.get("drugs_held") or 0) + SOLVE_TALLY[f"pathogen_rule={ds_result.get('pathogen_rule', 'unknown')}"] += 1 + SOLVE_TALLY["pathogens_held"] += int(ds_result.get("pathogens_held") or 0) SOLVE_TALLY[f"conserved_rule={ds_result.get('conserved_rule', 'unknown')}"] += 1 if ds_result.get("cycle_rule") == "balance: no pool table": raise SystemExit(f"{pathway_name}: DS_CYCLE_MODE=balance but the network has no pools.csv " @@ -1383,6 +1389,10 @@ def main(): if "inert" in rules and SOLVE_TALLY["drugs_held"] == 0: # An inert arm that held nothing measured the default model. raise SystemExit("DS_DRUG_MODE=inert but no solve held a drug node; the arm is the control.") + prules = sorted(k.split("=", 1)[1] for k in SOLVE_TALLY if k.startswith("pathogen_rule=")) + print(f"\nPathogens: rule {'/'.join(prules) or 'unknown'}, {SOLVE_TALLY['pathogens_held']} node-solves held (specs/048)") + if "inert" in prules and SOLVE_TALLY["pathogens_held"] == 0: + raise SystemExit("DS_PATHOGEN_MODE=inert but no solve held a pathogen node; the arm is the control.") yrules = sorted(k.split("=", 1)[1] for k in SOLVE_TALLY if k.startswith("cycle_rule=")) yc = sorted(k.split("=", 1)[1] for k in SOLVE_TALLY if k.startswith("cycle_carriers=")) print(f"\nCycles: rule {'/'.join(yrules) or 'unknown'}, {SOLVE_TALLY['cycle_pools_solved']} pool-solves, " diff --git a/cli/deltasignal.jl b/cli/deltasignal.jl index 82b831d..a9e85b9 100755 --- a/cli/deltasignal.jl +++ b/cli/deltasignal.jl @@ -207,6 +207,8 @@ function execute_parse_command(args) "containment" => network_containment_json(network), # And for DS_DRUG_MODE (specs/032); null when the bundle has no list. "drug_stids" => drug_stids_json(network), + # And for DS_PATHOGEN_MODE (specs/048). + "pathogen_stids" => pathogen_stids_json(network), "pools" => pools_json(network), "set_mappings" => Dict(set_id => Dict( "original_set_id" => mapping.original_set_id, @@ -394,7 +396,9 @@ function execute_solve_command(args) containment = containment_from_json(get(network_json, "containment", nothing)) drug_stids = drug_stids_from_json(get(network_json, "drug_stids", nothing)) pools = pools_from_json(get(network_json, "pools", nothing)) - network = ReactionNetwork(nodes, edges, set_mappings, cofactor_stids, containment, drug_stids, pools) + pathogen_stids = pathogen_stids_from_json(get(network_json, "pathogen_stids", nothing)) + network = ReactionNetwork(nodes, edges, set_mappings, cofactor_stids, containment, drug_stids, pools, + pathogen_stids) println("✓ Network loaded: $(length(nodes)) nodes, $(length(edges)) edges") # Load observations CSV diff --git a/scripts/run_arm.sh b/scripts/run_arm.sh index 1460ded..aa2215e 100755 --- a/scripts/run_arm.sh +++ b/scripts/run_arm.sh @@ -163,6 +163,7 @@ for gt in ("curator", "experimental"): m = re.search(r"^Pinned: .*$", text, re.M); rec[f"{gt}_pinned"] = m.group(0) if m else None m = re.search(r"^Converged: .*$", text, re.M); rec[f"{gt}_converged"] = m.group(0) if m else None m = re.search(r"^Drugs: .*$", text, re.M); rec[f"{gt}_drugs"] = m.group(0) if m else None + m = re.search(r"^Pathogens: .*$", text, re.M); rec[f"{gt}_pathogens"] = m.group(0) if m else None m = re.search(r"^Conserved: .*$", text, re.M); rec[f"{gt}_conserved"] = m.group(0) if m else None m = re.search(r"^Cycles: .*$", text, re.M); rec[f"{gt}_cycles"] = m.group(0) if m else None json.dump(rec, open(os.path.join(out, "ARM.json"), "w"), indent=2) diff --git a/src/DeltaSignal.jl b/src/DeltaSignal.jl index f244286..ae70157 100644 --- a/src/DeltaSignal.jl +++ b/src/DeltaSignal.jl @@ -50,6 +50,7 @@ export LogicNetworkEdge, ReactionParams, SolverResult export parse_logic_network, parse_uuid_mapping, parse_set_mappings, parse_complete_network, create_reaction_network export parse_containment, network_containment_json, containment_from_json export parse_drug_list, drug_stids_json, drug_stids_from_json, drug_mode, drug_uuids, pools_json, pools_from_json +export parse_pathogen_list, pathogen_stids_json, pathogen_stids_from_json, pathogen_mode, pathogen_uuids export parse_pools, cycle_mode, cycle_carriers, PoolPath export conserved_mode, conserved_uuids export convert_to_reaction_network, aggregate_to_pathway_view, validate_network_pathway_mapping diff --git a/src/api/server.jl b/src/api/server.jl index 46924b3..88eb4fb 100644 --- a/src/api/server.jl +++ b/src/api/server.jl @@ -334,8 +334,13 @@ function reaction_network_from_json(data)::DeltaSignal.ReactionNetwork catch throw(ArgumentError("`pools` must be null or the list /api/parse returns.")) end + pathogen_stids = try + DeltaSignal.pathogen_stids_from_json(get(data, :pathogen_stids, nothing)) + catch + throw(ArgumentError("`pathogen_stids` must be a list of stable ids or null.")) + end return DeltaSignal.ReactionNetwork(nodes_dict, edges, set_mappings, cofactor_stids, containment, - drug_stids, pools) + drug_stids, pools, pathogen_stids) end """ @@ -722,6 +727,8 @@ function parse_handler(req) # specs/032: the bundle's drug list (null when it has none), so a # parse -> POSTed solve runs the same model as a solve by network_id. "drug_stids" => DeltaSignal.drug_stids_json(network), + # specs/048: the same for the bundle's pathogen list. + "pathogen_stids" => DeltaSignal.pathogen_stids_json(network), # specs/039: the interconversion pool table (null when none), so a # parse -> POSTed solve runs the same cycle model as a solve by id. "pools" => DeltaSignal.pools_json(network), @@ -870,6 +877,9 @@ function solve_handler(req) # specs/032 (additive): "propagate", "inert", or "inert: no drug table". "drug_rule" => get(solver_result.diagnostics, "drug_rule", "unknown"), "drugs_held" => get(solver_result.diagnostics, "drugs_held", 0), + # specs/048 (additive): "propagate", "inert", or "inert: no pathogen table". + "pathogen_rule" => get(solver_result.diagnostics, "pathogen_rule", "unknown"), + "pathogens_held" => get(solver_result.diagnostics, "pathogens_held", 0), # specs/035 (additive): "off" or "inert", and how many nodes were held. "conserved_rule" => get(solver_result.diagnostics, "conserved_rule", "unknown"), "conserved_held" => get(solver_result.diagnostics, "conserved_held", 0), diff --git a/src/core/drugs.jl b/src/core/drugs.jl index 0138560..25aeff9 100644 --- a/src/core/drugs.jl +++ b/src/core/drugs.jl @@ -47,3 +47,46 @@ function drug_uuids(network::ReactionNetwork)::Set{String} end return out end + +""" +Pathogen-derived entities (specs/048), held exactly as drugs are. + +Reactome curates host-pathogen interactions inside human pathways: the +SARS-CoV-2 N:M:PDPK1 complex negatively regulates "PDPK1 phosphorylates AKT at +T308". A PDPK1 knockout zeroes that complex, de-represses the reaction, and the +AKT loop rails to 100x where the truth is DOWN. In a cell without the virus the +complex does not exist; held at baseline it is a constant that cancels in +fold-change. The generator lists the nodes in `pathogens.csv` (a leaf with a +species, none human; a complex with any such component; a set whose every +member is one). + + DS_PATHOGEN_MODE=propagate (default) — previous behaviour. + DS_PATHOGEN_MODE=inert — pathogen-derived nodes pinned at baseline, as + DS_DRUG_MODE=inert pins drugs. An explicit + observation still wins. +""" +const DS_VALID_PATHOGEN_MODES = Set(["propagate", "inert"]) + +function pathogen_mode()::String + value = get(ENV, "DS_PATHOGEN_MODE", "propagate") + if !(value in DS_VALID_PATHOGEN_MODES) + throw(ArgumentError( + "DS_PATHOGEN_MODE must be one of $(join(sort(collect(DS_VALID_PATHOGEN_MODES)), ", ")); got $(repr(value))")) + end + return value +end + +"""uuids in this network whose Reactome stable id the bundle declares pathogen-derived.""" +pathogen_uuids(network::ReactionNetwork)::Set{String} = _uuids_with_stid(network, network.pathogen_stids) + +function _uuids_with_stid(network::ReactionNetwork, stids)::Set{String} + out = Set{String}() + stids === nothing && return out + for (uuid, node) in network.nodes + rid = node.reactome_id + if rid !== nothing && rid in stids + push!(out, uuid) + end + end + return out +end diff --git a/src/io/tsv_parser.jl b/src/io/tsv_parser.jl index 7f719ad..5bd514e 100644 --- a/src/io/tsv_parser.jl +++ b/src/io/tsv_parser.jl @@ -89,13 +89,17 @@ struct ReactionNetwork # `nothing` = the bundle has no such file (DS_CYCLE_MODE=balance then reports # "balance: no pool table"); an empty vector = the file lists none. pools::Union{Nothing, Vector{CyclePool}} + # Pathogen-derived stable ids from the generator's `pathogens.csv` + # (specs/048), with the same `nothing` / empty distinction as drug_stids. + pathogen_stids::Union{Nothing, Set{String}} ReactionNetwork(nodes, edges, set_mappings, cofactor_stids = Set{String}(), containment = Dict{String, Set{String}}(), drug_stids = nothing, - pools = nothing) = - new(nodes, edges, set_mappings, cofactor_stids, containment, drug_stids, pools) + pools = nothing, + pathogen_stids = nothing) = + new(nodes, edges, set_mappings, cofactor_stids, containment, drug_stids, pools, pathogen_stids) end """ @@ -391,6 +395,7 @@ function parse_complete_network( stids = parse_cofactor_list(resolved; required = cofactor_path !== nothing) containment = parse_containment(logic_network_path) drugs = parse_drug_list(logic_network_path) + pathogens = parse_pathogen_list(logic_network_path) # A malformed pool table is an error when the solve would use it (the # default, DS_CYCLE_MODE=balance). Under DS_CYCLE_MODE=off it must not take # a pathway out of service, so it is reported and dropped. @@ -421,7 +426,7 @@ function parse_complete_network( end end return ReactionNetwork(network.nodes, network.edges, network.set_mappings, - Set{String}(), containment, drugs, pools) + Set{String}(), containment, drugs, pools, pathogens) end println("Found $(length(stids)) cofactor species declared by the bundle") @@ -450,7 +455,7 @@ function parse_complete_network( end return ReactionNetwork(network.nodes, network.edges, network.set_mappings, stids, - containment, drugs, pools) + containment, drugs, pools, pathogens) end """ @@ -458,12 +463,21 @@ Read the generator's `drugs.csv` beside a logic network (specs/032): the drug-derived stable ids present in this pathway. `nothing` when there is no such file (a bundle that predates it), an empty set when the file lists none. """ -function parse_drug_list(logic_network_path::String)::Union{Nothing, Set{String}} - path = joinpath(dirname(logic_network_path), "drugs.csv") +parse_drug_list(logic_network_path::String) = _parse_stid_list(logic_network_path, "drugs.csv", "drug") + +""" +Read the generator's `pathogens.csv` (specs/048): the pathogen-derived stable ids +present in this pathway, with `parse_drug_list`'s `nothing` / empty distinction. +""" +parse_pathogen_list(logic_network_path::String) = + _parse_stid_list(logic_network_path, "pathogens.csv", "pathogen") + +function _parse_stid_list(logic_network_path::String, file::String, what::String)::Union{Nothing, Set{String}} + path = joinpath(dirname(logic_network_path), file) isfile(path) || return nothing df = CSV.read(path, DataFrame; types = String) "stable_id" in names(df) || - throw(ArgumentError("$(path) has no `stable_id` column; it is not a generator drug list")) + throw(ArgumentError("$(path) has no `stable_id` column; it is not a generator $(what) list")) return Set(String(x) for x in df.stable_id if !ismissing(x) && !isempty(strip(x))) end @@ -603,9 +617,18 @@ function _pools_from_json(raw)::Vector{CyclePool} end """Inverse of `drug_stids_json`.""" -function drug_stids_from_json(raw)::Union{Nothing, Set{String}} +drug_stids_from_json(raw) = _stids_from_json(raw, "drug_stids") + +"""JSON form of `pathogen_stids` (specs/048), as `drug_stids_json`.""" +pathogen_stids_json(network::ReactionNetwork) = + network.pathogen_stids === nothing ? nothing : sort(collect(network.pathogen_stids)) + +"""Inverse of `pathogen_stids_json`.""" +pathogen_stids_from_json(raw) = _stids_from_json(raw, "pathogen_stids") + +function _stids_from_json(raw, field::String)::Union{Nothing, Set{String}} raw === nothing && return nothing - raw isa AbstractVector || throw(ArgumentError("drug_stids must be a list of stable ids or null")) + raw isa AbstractVector || throw(ArgumentError("$(field) must be a list of stable ids or null")) return Set(String.(collect(raw))) end diff --git a/src/solvers/steady_state.jl b/src/solvers/steady_state.jl index 877c246..38f4634 100644 --- a/src/solvers/steady_state.jl +++ b/src/solvers/steady_state.jl @@ -118,6 +118,11 @@ function solve_steady_state( drugs = dmode == "inert" ? drug_uuids(network) : Set{String}() drug_rule = dmode == "propagate" ? "propagate" : network.drug_stids === nothing ? "inert: no drug table" : "inert" + # specs/048: pathogen-derived nodes, the same hold for an uninfected cell. + pmode = pathogen_mode() + pathogens = pmode == "inert" ? pathogen_uuids(network) : Set{String}() + pathogen_rule = pmode == "propagate" ? "propagate" : + network.pathogen_stids === nothing ? "inert: no pathogen table" : "inert" # specs/035: loop participants released unchanged by their loop, held at # baseline like cofactors. Resolved on the network as given (before bridges). cmode = conserved_mode() @@ -133,7 +138,8 @@ function solve_steady_state( @info "Adding $(length(bridges)) silo bridge edge(s)" max_reach=silo_bridge_max_reach() network = ReactionNetwork(network.nodes, vcat(network.edges, bridges), network.set_mappings, network.cofactor_stids, - network.containment, network.drug_stids, network.pools) + network.containment, network.drug_stids, network.pools, + network.pathogen_stids) end # Convert network to reactions @@ -184,6 +190,16 @@ function solve_steady_state( drugs_held = length(dpins) observations = merge(observations, dpins) end + pathogens_held = 0 + if !isempty(pathogens) + # The drug rule's pin precedence: an observation that pins wins. + ppins = Dict{String, Tuple{Float64, Float64}}( + u => (network.nodes[u].baseline * 100.0, 1.0) + for u in pathogens + if !(haskey(observations, u) && observations[u][2] > OBS_CONFIDENCE_TOL)) + pathogens_held = length(ppins) + observations = merge(observations, ppins) + end conserved_held = 0 if !isempty(conserved) # An observation that PINS wins (a perturbed GAP such as NF1 stays @@ -254,6 +270,8 @@ function solve_steady_state( result.diagnostics["cycle_carriers"] = ymode == "balance" ? (cycle_carriers() ? "on" : "off") : "off" result.diagnostics["drug_rule"] = drug_rule result.diagnostics["drugs_held"] = drugs_held + result.diagnostics["pathogen_rule"] = pathogen_rule + result.diagnostics["pathogens_held"] = pathogens_held result.diagnostics["conserved_rule"] = cmode result.diagnostics["conserved_held"] = conserved_held # specs/040: which self-feedback rules ran. The counts come from the solve. diff --git a/test/test_drugs.jl b/test/test_drugs.jl index 4178f59..89bca48 100644 --- a/test/test_drugs.jl +++ b/test/test_drugs.jl @@ -176,4 +176,120 @@ end @test_throws ArgumentError DS.drug_stids_from_json("R-D") end +# specs/048: pathogen-derived nodes (pathogens.csv) under DS_PATHOGEN_MODE, the +# same hold on its own switch. Fixture: D is a viral protein, C = X:D inhibits +# the reaction making T (the N:M:PDPK1 shape). +function pfixture(pathogen_stids; drug_stids = nothing) + f = fixture(drug_stids) + DS.ReactionNetwork(f.nodes, f.edges, f.set_mappings, f.cofactor_stids, f.containment, + f.drug_stids, f.pools, pathogen_stids) +end + +function psolve(net; mode, drug = nothing, x = 2.0, extra = Dict{String, Tuple{Float64, Float64}}()) + with_env("DS_PATHOGEN_MODE" => mode) do + solve(net; mode = drug, x = x, extra = extra) + end +end + +@testset "pathogens (specs/048)" begin + @testset "default is propagate, byte-identical to a network with no pathogen list" begin + with_env("DS_PATHOGEN_MODE" => nothing) do + @test DS.pathogen_mode() == "propagate" + end + a = psolve(pfixture(Set(["R-C", "R-D"])); mode = nothing) + b = psolve(pfixture(nothing); mode = nothing) + @test a.node_activities == b.node_activities + @test a.diagnostics["pathogen_rule"] == "propagate" + @test a.diagnostics["pathogens_held"] == 0 + @test fold(a, "T") ≈ 1.0 rtol = 1e-6 + # the pre-existing 7-argument constructor leaves the list absent + @test fixture(nothing).pathogen_stids === nothing + end + @testset "inert holds pathogen-derived nodes at baseline and T follows X" begin + for x in (2.0, 0.5, 0.0, 50.0) + r = psolve(pfixture(Set(["R-C", "R-D"])); mode = "inert", x = x) + @test fold(r, "C") ≈ 1.0 rtol = 1e-9 + @test fold(r, "D") ≈ 1.0 rtol = 1e-9 + @test fold(r, "T") ≈ min(x, 100.0) rtol = 1e-6 + @test r.diagnostics["pathogen_rule"] == "inert" + @test r.diagnostics["pathogens_held"] == 2 + @test r.diagnostics["drugs_held"] == 0 + end + end + @testset "independent of the drug rule" begin + # a node listed only as a drug is not held by DS_PATHOGEN_MODE, and vice versa + r = psolve(pfixture(nothing; drug_stids = Set(["R-C"])); mode = "inert") + @test r.diagnostics["pathogen_rule"] == "inert: no pathogen table" + @test fold(r, "C") ≈ 2.0 rtol = 1e-6 + r = psolve(pfixture(Set(["R-C"]); drug_stids = nothing); mode = nothing, drug = "inert") + @test r.diagnostics["drug_rule"] == "inert: no drug table" + @test r.diagnostics["pathogens_held"] == 0 + @test fold(r, "C") ≈ 2.0 rtol = 1e-6 + end + @testset "a missing pathogen table is reported, not pretended" begin + r = psolve(pfixture(nothing); mode = "inert") + @test r.diagnostics["pathogen_rule"] == "inert: no pathogen table" + @test r.diagnostics["pathogens_held"] == 0 + r = psolve(pfixture(Set{String}()); mode = "inert") + @test r.diagnostics["pathogen_rule"] == "inert" + @test r.diagnostics["pathogens_held"] == 0 + end + @testset "an explicit observation wins; a confidence-0 one does not" begin + r = psolve(pfixture(Set(["R-C", "R-D"])); mode = "inert", x = 1.0, extra = Dict("C" => (4.0, 1.0))) + @test fold(r, "C") ≈ 4.0 rtol = 1e-9 + @test r.diagnostics["pathogens_held"] == 1 + r = psolve(pfixture(Set(["R-C", "R-D"])); mode = "inert", x = 2.0, extra = Dict("C" => (4.0, 0.0))) + @test fold(r, "C") ≈ 1.0 rtol = 1e-9 + @test r.diagnostics["pathogens_held"] == 2 + end + @testset "a typo in the mode is an error" begin + for bad in ("Inert", "inert ", "on", "") + with_env("DS_PATHOGEN_MODE" => bad) do + @test_throws ArgumentError DS.pathogen_mode() + end + end + end + @testset "pathogens.csv: both parse return paths, the silo-bridge rebuild, JSON" begin + mktempdir() do dir + write(joinpath(dir, "logic_network.csv"), + "source_id,target_id,pos_neg,and_or,edge_type,stoichiometry\n" * + "u-d,u-c,pos,and,assembly,1\nu-x,u-c,pos,and,assembly,1\n") + write(joinpath(dir, "stid_to_uuid_mapping.csv"), + "uuid,stable_id\nu-d,R-COV-9\nu-x,R-HSA-1\nu-c,R-HSA-2\n") + ln, um = joinpath(dir, "logic_network.csv"), joinpath(dir, "stid_to_uuid_mapping.csv") + @test DS.parse_pathogen_list(ln) === nothing + @test DS.parse_complete_network(ln, um).pathogen_stids === nothing + write(joinpath(dir, "pathogens.csv"), "stable_id,schema_class,name,reactome_release\n") + @test DS.parse_pathogen_list(ln) == Set{String}() + write(joinpath(dir, "pathogens.csv"), + "stable_id,schema_class,name,reactome_release\nR-COV-9,EWAS,M,97\nR-HSA-2,Complex,N:M:PDPK1,97\n") + net = DS.parse_complete_network(ln, um) + @test net.pathogen_stids == Set(["R-COV-9", "R-HSA-2"]) + @test DS.pathogen_uuids(net) == Set(["u-d", "u-c"]) + write(joinpath(dir, "cofactors.csv"), + "stable_id,molecule,chebi_id,name,in_network,reactome_release\nR-HSA-1,ATP,1,x,1,97\n") + @test DS.parse_complete_network(ln, um).pathogen_stids == Set(["R-COV-9", "R-HSA-2"]) + write(joinpath(dir, "pathogens.csv"), "id\nR-COV-9\n") + @test_throws ArgumentError DS.parse_pathogen_list(ln) + end + nodes = Dict(node.(["X", "D", "C", "W", "T"])) + nodes["S1"] = DS.NetworkNode("S1", "R-S", "protein", nothing, "S1", BL) + nodes["S2"] = DS.NetworkNode("S2", "R-S", "protein", nothing, "S2", BL) + f = fixture(nothing) + net = DS.ReactionNetwork(nodes, vcat(f.edges, [edge("X", "S1", true, "input"), edge("S2", "T", true, "input")]), + f.set_mappings, f.cofactor_stids, f.containment, nothing, nothing, Set(["R-C", "R-D"])) + with_env("DS_SILO_BRIDGE_MAX_REACH" => "50") do + @test !isempty(DS.silo_bridge_edges(net)) + r = psolve(net; mode = "inert") + @test r.diagnostics["pathogens_held"] == 2 + @test fold(r, "C") ≈ 1.0 rtol = 1e-9 + end + @test DS.pathogen_stids_json(pfixture(nothing)) === nothing + @test DS.pathogen_stids_json(pfixture(Set(["R-D", "R-C"]))) == ["R-C", "R-D"] + @test DS.pathogen_stids_from_json(nothing) === nothing + @test DS.pathogen_stids_from_json(["R-D"]) == Set(["R-D"]) + @test_throws ArgumentError DS.pathogen_stids_from_json("R-D") + end +end + end From 3b856a864674d1c21ea98ab122f38f0b1e1f2f89 Mon Sep 17 00:00:00 2001 From: Adam Wright <668955+adamjohnwright@users.noreply.github.com> Date: Sat, 10 Oct 2026 08:24:11 -0400 Subject: [PATCH 04/10] specs/048 results: pathogen-inert misses the pathway floor (PIP3 +7 exp, DDX58/IFIH1 -11 held-out); bind changes no prediction, adopt on faithfulness Co-Authored-By: Claude Opus 5.5 (1M context) --- specs/048-experimental-levers/research.md | 136 ++++++++++++++++++++++ 1 file changed, 136 insertions(+) create mode 100644 specs/048-experimental-levers/research.md diff --git a/specs/048-experimental-levers/research.md b/specs/048-experimental-levers/research.md new file mode 100644 index 0000000..f3429ce --- /dev/null +++ b/specs/048-experimental-levers/research.md @@ -0,0 +1,136 @@ +# specs/048 research: arm results + +All arms ran on solver commit `4cb152d` with `scripts/run_arm.sh`, protocol +`DS_PIN_SCOPE=root_cycle`. Scoring is `bench/analysis/arm_compare.py`, paired on +(pathway, gene, direction, readout), because a rebuild re-mints every uuid. + +| arm | build | server override | results | +|---|---|---|---| +| canonical | `20261009-2307_132d4b8` | — | `results/0f0a11e` | +| `pathctl` | `20261010-0259_a43e20d_path` | `DS_PATHOGEN_MODE=propagate` | `results/4cb152d/pathctl` | +| `path` | `20261010-0259_a43e20d_path` | `DS_PATHOGEN_MODE=inert` | `results/4cb152d/path` | +| `bind` | `20261010-0341_a43e20d_bind` (`LNG_BIND_STOICH=1`) | — | `results/4cb152d/bind` | + +Valid cases are identical in every arm: curator 23,511 (18,481 held-out, 5,030 +tuning), experimental 845. Convergence is identical in every arm: 1,654 of +1,725 curator solves, 215 of 244 experimental solves. + +## Control: `pathctl` reproduces canonical + +- Every one of 24,100 curator and 849 experimental cases has the same + prediction, validity and category as canonical. +- 9 curator values differ by more than 1e-6. Eight are DSB Repair + overexpressions reading 75–89× that move by at most 0.0016 (relative 2e-5). + The ninth is one EGFR value at 0.2965 that moves by 2e-6. None is near a + threshold. +- These shifts are not traced. The build's networks have the same node and + edge counts as canonical. Predictions match, which is what the amendment + required, so `path` is read against `pathctl`. + +## Lever A, arm `path`: NOT ADOPTED (fails the pathway floor by one case) + +| axis | split | `pathctl` → `path` | net (fixed/broke) | McNemar p | +|---|---|---|---|---| +| experimental | all (tuning) | 69.11% → 69.94% | **+7** (7/0) | 0.016 | +| curator | held-out | 88.99% → 88.93% | **−11** (1/12) | 0.0034 | +| curator | tuning | 76.14% → 76.42% | +14 (14/0) | 0.00012 | +| curator | all | 86.24% → 86.26% | +3 (15/12) | 0.7 | + +- The servers held 128 node-solves (experimental) and 14,642 node-solves + (curator). +- Two pathways moved, on 10 perturbations: + - PIP3, all PDPK1 knockout: +7 experimental, +14 curator. + - DDX58/IFIH1: −11 curator held-out, across 9 perturbations. + +**Predictions:** +- **A-P1 met.** All 7 PDPK1-knockout DOWN readouts go from 100× to 0×. +- **A-P2 met.** The 3 PDPK1-overexpression UP readouts stay UP; they read + 100× where before they read 64.5×. +- **A-P3 met.** Experimental net is +7, all in PIP3, and no other + experimental case changes. +- **A-P4 met.** Held-out net is −11, inside ±15. + +**Decision rule:** +- Experimental net > 0: yes. +- Held-out ≥ −15: yes. +- No pathway loses more than 10: **no.** DDX58/IFIH1 loses 11 (1 fixed, 12 + broken). + +By the pre-registered rule, lever A is not adopted. `DS_PATHOGEN_MODE` stays +in the solver at `propagate` (the default), as a measured record, like +`DS_DRUG_MODE`. + +### Traced: the DDX58/IFIH1 loss + +- **What is held.** `R-HSA-168928/pathogens.csv` has 633 rows over 45 base + entities. 33 of the 45 are human complexes that contain a viral component. + Most of them are the RIG-I/MDA5 signalling platform: + - `viral dsRNA:IFIH1, viral dsRNA:K63polyUb-DDX58:MAVS:...` with TRAF3, + TBK1/IKBKE, IRF3/IRF7, RIPK1:FADD:CASP8/10, and IKK; + - R-HSA-918199, 933470, 933479, 9705147, 8983916 and 933478 among them, + with up to 48 variant rows each. +- **Why it breaks.** Every complex on the route from a host signalling + protein to its readout contains the viral dsRNA ligand. Holding those + complexes cuts every host perturbation that runs through the signalosome. + - CYLD, IKBKB, MAP3K1 and CASP8, knocked out or overexpressed, all read + exactly 1.0 at readouts 909690, 177673, 877351 and 933478, where they + were correct before. + - The one fix is a CREBBP knockout at 877351. It goes from 3.5× to 0.5×, + because the `CREBBP:NS1` antagonist complex is now held. +- **The two curations differ.** + - In PIP3, the pathogen is an antagonist: SARS-CoV-2 N:M sequesters a host + kinase. A cell without the virus has no such complex, and holding it is + faithful. + - In DDX58/IFIH1, the viral RNA is the stimulus the pathway exists to + sense, and the curator ground truth assumes it is present. A-P4 named + this risk and did not predict its direction. +- **A narrower rule would separate the two.** It would hold a complex only + when its pathogen part is a protein bound to a host protein, not a + nucleic-acid ligand. That rule is post hoc. Its experimental upside is + capped at the same +7, and it would need its own pre-registration and arm. + It is not run here. + +## Arm `bind` (B3, `LNG_BIND_STOICH=1`): ADOPT on faithfulness + +| axis | split | canonical → `bind` | net | +|---|---|---|---| +| curator | held-out | 88.99% → 88.99% | 0 (0/0) | +| curator | tuning | 76.14% → 76.14% | 0 (0/0) | +| experimental | all | 69.11% → 69.11% | 0 (0/0) | + +**Structural checks:** +1. **Met.** The build is complete: 213,171 nodes and 516,528 edges, against + 213,061 and 516,244 for canonical. +2. **Met.** Coverage is identical on both axes. +3. **Met.** In every copy of R-HSA-5675376, p-MAPK1 and p-MAPK3 go to their + monomers and the phosphorylated dimers to their dimers. +4. **Counted before scoring.** Binding ties fall from 64 to 58 catalog-wide. + - Edges change in 10 pathways: +84, +16, +40, +24, +72, +6, +12, +4 (RAF), + +16 and +10. + - Stable-id sets are identical in all of them. + +**Predictions:** +- **bind-P1** registered no direction, and nothing moved. +- **bind-P2 missed.** RAF values move, but no prediction does: + - 28 experimental values change, all in RAF. + - The largest change is BRAF knockout at readout 1268261: 70.38× → 70.22×. + - The 12 knockout→UP cases still read 36–70× through + `R-HSA-169291::pool`. + - Correcting the binding closes the curated MAPK ⇄ p-MAPK cycle, but the + knockout→UP class does not depend on it. Two pools share 246 nodes, and + that is still unsolved. +- 226 curator values move, none across a threshold. Most are IFN-γ values + near zero that change in the sixth decimal. + +**Decision rule** (held-out ≥ −15 and experimental ≥ −15): met at 0 and 0. +B3 is adopted as a faithfulness fix. That means `LNG_BIND_STOICH` defaults +on in the generator (an LNG PR, which Adam merges). Canonical numbers do not +change. + +## What this leaves + +- The RAF knockout→UP class (12 of 21 RAF failures) needs pools that share + intermediates. That is the lever B design question in Amendment 1. B1 and + B2 are on `wip/048-mapk-pools`. +- PDPK1 (+7 experimental) is recoverable only by a pathogen rule that spares + sensor ligands. That rule would need a new registration. From 995703c3afaf156f7108fdcde34445a90c3fb212 Mon Sep 17 00:00:00 2001 From: Adam Wright <668955+adamjohnwright@users.noreply.github.com> Date: Sat, 10 Oct 2026 11:42:37 -0400 Subject: [PATCH 05/10] specs/048: DDX58/IFIH1 loss traced case by case and verified in Neo4j; bind default on in LNG #110 Co-Authored-By: Claude Opus 5.5 (1M context) --- specs/048-experimental-levers/research.md | 28 +++++++++++++++++++++-- 1 file changed, 26 insertions(+), 2 deletions(-) diff --git a/specs/048-experimental-levers/research.md b/specs/048-experimental-levers/research.md index f3429ce..3ad9afc 100644 --- a/specs/048-experimental-levers/research.md +++ b/specs/048-experimental-levers/research.md @@ -77,6 +77,30 @@ in the solver at `propagate` (the default), as a measured record, like were correct before. - The one fix is a CREBBP knockout at 877351. It goes from 3.5× to 0.5×, because the `CREBBP:NS1` antagonist complex is now held. +- **Case by case** + (`~/deltasignal-catalogs/analysis/048/ddx58_trace.py`, routes in the + `_path` network). All 12 broken cases fail in one of two ways: + - **The readout itself is held (4 cases).** IKBKB and CASP8, up and down, + at readout 933478. That readout is + `dsRNA:RIG-I/MDA5:...:IKK complex`, which contains the viral ligand, so it + is pinned at 1.0 whatever the perturbation does. + - **Every route passes a held node (8 cases).** CYLD up and down at + 909690, 177673 and 877351, and MAP3K1 up and down at 177673. + - Shortest routes before the pin: 19–31 nodes for CYLD, 9 for MAP3K1. + - Routes that avoid held nodes: none. + - On the CYLD routes, the first held node is `2x DDX58 ligand:2x + DDX58:2xATP` (R-HSA-168906), RIG-I bound to viral RNA. On the MAP3K1 + route it is `dsRNA:DDX58/IFIH1:MAVS:TRAF2/TRAF6:MEKK1` (R-HSA-933482). + - **The one fix.** CREBBP knockout at 877351 has a 7-node route with no + held node. It moves from 3.5× to 0.5× because the `CREBBP:NS1` + antagonist is held rather than knocked out with CREBBP. +- **Neo4j confirms the curation** (`ddx58_neo4j.py`, release 97): + - "DDX58 ligand" (R-NUL-9013905) holds only viral RNAs: Rotavirus RNA, HCV + 5'-ppp poly-U/UC RNA, and the Influenza A dsRNA intermediate. + - "IFIH1 ligand" (R-NUL-9038422) holds Measles, RSV A and Influenza A + dsRNA. + - Neither set has a host member. Reactome curates no form of the RIG-I/MDA5 + platform without the virus, so this is not a generator artefact. - **The two curations differ.** - In PIP3, the pathogen is an antagonist: SARS-CoV-2 N:M sequesters a host kinase. A cell without the virus has no such complex, and holding it is @@ -123,8 +147,8 @@ in the solver at `propagate` (the default), as a measured record, like near zero that change in the sixth decimal. **Decision rule** (held-out ≥ −15 and experimental ≥ −15): met at 0 and 0. -B3 is adopted as a faithfulness fix. That means `LNG_BIND_STOICH` defaults -on in the generator (an LNG PR, which Adam merges). Canonical numbers do not +B3 is adopted as a faithfulness fix. `LNG_BIND_STOICH` now defaults on in +generator PR #110 (commit 0123fbb), which Adam merges. Canonical numbers do not change. ## What this leaves From e2daa7eb6bdaac91f82deddbb5da552c951ce5a5 Mon Sep 17 00:00:00 2001 From: Adam Wright <668955+adamjohnwright@users.noreply.github.com> Date: Sat, 10 Oct 2026 11:54:07 -0400 Subject: [PATCH 06/10] specs/048 amendment 2: pre-register pathogen-protein-only rule (arm pathp) before implementation Co-Authored-By: Claude Opus 5.5 (1M context) --- specs/048-experimental-levers/spec.md | 62 +++++++++++++++++++++++++++ 1 file changed, 62 insertions(+) diff --git a/specs/048-experimental-levers/spec.md b/specs/048-experimental-levers/spec.md index 640ca22..f1b77f1 100644 --- a/specs/048-experimental-levers/spec.md +++ b/specs/048-experimental-levers/spec.md @@ -219,3 +219,65 @@ added, the release step ends at p-MAPK1. - **Decision rule:** adopt on faithfulness if held-out ≥ −15 and experimental ≥ −15 (the specs/046 rule for a representation fix). A loss beyond either floor is traced first. + +## Amendment 2 (2026-10-10, after arm `path`, before arm `pathp`): pathogen proteins only + +**This is post hoc.** Arm `path` failed the pathway floor by one case, and +I traced the loss before writing this rule (`research.md`). The rule's +definition comes from Reactome's molecule classes, not from the cases. Its +predictions, though, are written knowing what `path` did, so they test +whether the trace is right, not whether the idea is. + +**What `path` showed.** Two curations of a pathogen point in opposite +directions: +- an **antagonist**: a viral protein bound to a host protein, such as + SARS-CoV-2 N:M:PDPK1 in PIP3; +- a **sensed ligand**: viral RNA that RIG-I/MDA5 exist to detect. Every + signalling complex in DDX58/IFIH1 carries it, and the curator ground truth + assumes it is present. + +**Neo4j census** (`analysis/048/pathogen_leaf_kinds.py`, release 97) of the +20 pathogen leaves in the catalog: +- 15 are proteins: `EntityWithAccessionedSequence` on a + `ReferenceGeneProduct`, one of them on a `ReferenceIsoform`. +- 5 are viral RNAs: `GenomeEncodedEntity` with no reference entity, from + Influenza A, HCV, Measles, Rotavirus and RSV A. +- The RNAs appear only in DDX58/IFIH1 (R-HSA-168928). + +**Rule** (generator `LNG_PATHOGEN_PROTEIN=1`, default 0). This is the +specs/048 pathogen rule with one change: +- A leaf counts only when it is non-human **and** is a protein, meaning an + EWAS whose reference entity is a `ReferenceGeneProduct`, including + `ReferenceIsoform`. +- Complexes count when any component counts; sets count when every member + does. +- The solver is unchanged: `DS_PATHOGEN_MODE=inert` holds whatever + `pathogens.csv` lists. + +**Known weakness.** The rule cannot tell an antagonist from a sensed +protein ligand. NK-cell receptors bound to haemagglutinin (198933) are held +under both rules. They moved no case under `path`. + +**Arms.** One build, `--variant pathp --env LNG_PATHOGEN_PROTEIN=1`. Its +networks must match the `_path` build in node and edge counts per pathway, +and only DDX58/IFIH1's `pathogens.csv` may differ. Two arms run on it: +- `pathpctl` (`propagate`) must reproduce `pathctl`'s predictions exactly; +- `pathp` (`inert`). + +**Predictions** (`pathp` against `pathpctl`): +- **P2-1.** Experimental is the same as `path`: +7 (7/0), all PDPK1 + knockout in PIP3. +- **P2-2.** In DDX58/IFIH1, the 12 cases `path` broke (CYLD, IKBKB, MAP3K1 + and CASP8) keep their `pathctl` predictions. + - The CREBBP knockout fix at 877351 stays, because `CREBBP:NS1` is a + protein complex and is still held. + - DDX58/IFIH1 net is +1. +- **P2-3.** Curator held-out net is +1 and tuning is +14 (PIP3). No other + pathway moves. + +**Decision rule:** as for lever A. Adopt if: +- experimental net > 0; +- held-out ≥ −15; +- no pathway loses more than 10 on either axis. + +Any miss is traced to a case. From ec38eb9a905b4ed3051dfd4114b0283573824e40 Mon Sep 17 00:00:00 2001 From: Adam Wright <668955+adamjohnwright@users.noreply.github.com> Date: Sat, 10 Oct 2026 12:46:46 -0400 Subject: [PATCH 07/10] specs/048 amendment 2 result: pathogen-protein-only rule meets every prediction (exp +7, held-out +1, nothing broken) - adopt; combination with bind still to measure Co-Authored-By: Claude Opus 5.5 (1M context) --- specs/048-experimental-levers/research.md | 58 +++++++++++++++++++++++ 1 file changed, 58 insertions(+) diff --git a/specs/048-experimental-levers/research.md b/specs/048-experimental-levers/research.md index 3ad9afc..abf7dc5 100644 --- a/specs/048-experimental-levers/research.md +++ b/specs/048-experimental-levers/research.md @@ -158,3 +158,61 @@ change. B2 are on `wip/048-mapk-pools`. - PDPK1 (+7 experimental) is recoverable only by a pathogen rule that spares sensor ligands. That rule would need a new registration. + +## Amendment 2, arm `pathp` (pathogen proteins only): ADOPT + +- **Build:** `20261010-1155_f58ea98_pathp`, with `LNG_PATHOGEN_PROTEIN=1` + and `LNG_BIND_STOICH=0`. + - It is complete, with 213,061 nodes and 516,244 edges, the same as + `_path`. Node and edge counts match in every pathway, and `drugs.csv` is + identical. + - Only DDX58/IFIH1's `pathogens.csv` changes: 633 rows → 13, none of them + new. What remains is SARS-CoV-2 M, N, 9b and nsp13 and their host + complexes, and RSV NS1 with IRF3:NS1 and CREBBP:NS1. +- **Solver:** commit `e2daa7e`; `src/` is unchanged since `4cb152d`. +- **Results:** `results/e2daa7e/{pathpctl,pathp}`. +- **Held:** 1,754 curator node-solves (14,642 under `path`) and 128 + experimental, the same as `path`. + +**Control.** `pathpctl` reproduces `pathctl` on every prediction, validity +and category. 10 curator values shift by more than 1e-6: +- the same DSB Repair readout, 5693527, by up to 0.026 (RNF168, 71.34 → + 71.32); +- the same EGFR value at 0.2965. + +These are the shifts that canonical → `pathctl` showed. Each rebuild of +identical networks moves them, so something in that DSB component still +depends on labels, contrary to specs/013. It is small and far from any +threshold, but not traced; it is an open item. + +| axis | split | `pathpctl` → `pathp` | net (fixed/broke) | McNemar p | +|---|---|---|---|---| +| experimental | all (tuning) | 69.11% → 69.94% | **+7** (7/0) | 0.016 | +| curator | held-out | 88.99% → 89.00% | **+1** (1/0) | 1 | +| curator | tuning | 76.14% → 76.42% | +14 (14/0) | 0.00012 | +| curator | all | 86.24% → 86.31% | +15 (15/0) | 6.1e-05 | + +Two pathways moved, on two perturbations: PDPK1 knockout in PIP3, and +CREBBP knockout in DDX58/IFIH1. Nothing breaks on either axis. + +**Predictions:** +- **P2-1 met.** Experimental net is +7 (7/0), the same 7 PDPK1-knockout + DOWN readouts as `path`. +- **P2-2 met.** None of the 12 DDX58/IFIH1 cases that `path` broke changes. + The CREBBP fix at 877351 stays (3.5× → 0.5×), and DDX58/IFIH1 net is +1. +- **P2-3 met.** Held-out net is +1, tuning +14, and no other pathway moves. + +**Decision rule:** +- experimental > 0: yes; +- held-out ≥ −15: yes; +- no pathway loses more than 10: yes, nothing loses. + +**Adopted.** This means: +- `LNG_PATHOGEN_PROTEIN=1` and `DS_PATHOGEN_MODE=inert` become the defaults; +- `bind` (B3) is adopted too, so by the decision rule the combination is + measured once before it becomes canonical. + +**Caveat.** The rule was written after `path`'s trace, so its predictions +confirm the trace rather than test the idea blind. The gain rests on one +curated SARS-CoV-2 complex in PIP3, and the experimental +7 is one +perturbation. From 2a3db749425538012fbc236cf572077880360eef Mon Sep 17 00:00:00 2001 From: Adam Wright <668955+adamjohnwright@users.noreply.github.com> Date: Sat, 10 Oct 2026 12:55:37 -0400 Subject: [PATCH 08/10] DS_PATHOGEN_MODE=inert by default (specs/048 amendment 2 adopted: exp +7, held-out +1, 0 broken); combo arm registered; API.md documents pathogen_stids/pathogen_rule Co-Authored-By: Claude Opus 5.5 (1M context) --- CLAUDE.md | 24 ++++++++++++++++++++++- docs/API.md | 7 +++++++ specs/048-experimental-levers/research.md | 16 +++++++++++++++ src/core/drugs.jl | 11 ++++++++--- test/test_drugs.jl | 16 ++++++++++----- 5 files changed, 65 insertions(+), 9 deletions(-) diff --git a/CLAUDE.md b/CLAUDE.md index a2bb57b..f1c97af 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -348,6 +348,20 @@ env vars only override for benchmark sweeps: nodes (a Reactome `Drug`, a complex containing one, a set that is all drugs), listed by the generator's `drugs.csv`, held at baseline like cofactors, for a cell without the drug. The solve reports `drug_rule` and `drugs_held`. +- `DS_PATHOGEN_MODE=inert` (**default since 2026-10-10**, specs/048 amendment + 2) | `propagate`: nodes listed in the generator's `pathogens.csv` are held + at baseline, as drugs are, for a cell without the pathogen. An observation + still wins. + - Under the generator's `LNG_PATHOGEN_PROTEIN` (default on), the list holds + only entities that carry a pathogen PROTEIN (an EWAS on a + `ReferenceGeneProduct`): a protein leaf, a complex with such a component, + or a set whose members all are, e.g. SARS-CoV-2 N:M:PDPK1. + - Viral RNAs are not listed. They are the ligand DDX58/IFIH1 senses, and + holding them cost 12 curator cases (arm `path`). + - Against `propagate`: experimental +7 (all PDPK1 knockout), held-out +1, + nothing broken. + - The solve reports `pathogen_rule` and `pathogens_held`. A bundle without + the table reports `inert: no pathogen table`, and the bench refuses it. - `DS_SET_POOL_MODE=product` (default) | `extreme` | `geomean` | `mean` | `max` (specs/033): how a set POOL node (the generator's `LNG_SET_POOL`: one node per set-valued catalyst or regulator, fed by `set_member` edges) combines its @@ -512,6 +526,14 @@ for the last feature that touched it rather than re-deriving from the code. matches or beats it in 8 of 10 pathways: **the gap is network construction** (RAF, Mitotic G1, Cell Cycle Checkpoints, TP53), traced to five mechanism classes. `bench/analysis/experimental_gap.py` reproduces it. +- `specs/048-experimental-levers/` — two levers from 047. + - **Pathogen-derived nodes held at baseline: adopted**, as the protein-only + rule of amendment 2. The first rule also held viral RNA and lost 11 in + DDX58/IFIH1. + - **Homodimer binding fix (`LNG_BIND_STOICH`): adopted** on faithfulness; + it moves no prediction. + - Pools that share intermediates (the RAF knockout→UP class) are + unresolved; that work is on `wip/048-mapk-pools`. - `specs/009-solver-defaults/` — the one-variable-at-a-time re-measurement behind the `DS_*` defaults above (cited in that section too). - `.specify/memory/constitution.md` — project principles the specs are @@ -538,7 +560,7 @@ other seven execute code and print output. | `test/test_and_curves.jl` | 71 | | | `test/test_scc_break_roles.jl` | 56 | | | `test/test_self_inhibition.jl` | 82 | specs/022, added 2026-09-25 | -| `test/test_drugs.jl` | 121 | specs/032, added 2026-09-26; pathogens (specs/048) 2026-10-10 | +| `test/test_drugs.jl` | 125 | specs/032, added 2026-09-26; pathogens (specs/048) 2026-10-10 | | `test/test_set_pool.jl` | 79 | specs/033, added 2026-09-26; `variant_pool` (specs/046) 2026-10-09 | | `test/test_conserved.jl` | 15 | specs/035, added 2026-09-26 | | `test/test_cycles.jl` | 236 | specs/039, added 2026-09-27 | diff --git a/docs/API.md b/docs/API.md index 53c09d1..72ec0c1 100644 --- a/docs/API.md +++ b/docs/API.md @@ -108,6 +108,9 @@ are what the solver needs to run the default model; send them back with an inline network. `drug_stids` (specs/032, additive) lists the bundle's drug-derived stable ids, or is `null` when the bundle has no `drugs.csv`; send it back too, or `DS_DRUG_MODE=inert` reports `"inert: no drug table"`. +`pathogen_stids` (specs/048, additive) is the same for the bundle's +`pathogens.csv`. Without it, the default `DS_PATHOGEN_MODE=inert` reports +`"inert: no pathogen table"` and holds nothing. `pools` (specs/039, additive) is the interconversion-pool table, or `null` when the bundle has no `pools.csv`. Send it back with an inline network, or the default `DS_CYCLE_MODE=balance` reports `"balance: no pool table"` and loops @@ -166,6 +169,10 @@ Response: (default), `"inert"`, or `"inert: no drug table"`, and how many drug-derived nodes this solve held at baseline. Under `inert` a drug participates at fold 1.0 and never carries a perturbation; an explicit observation still wins. +- **`pathogen_rule`** / **`pathogens_held`** (additive, specs/048): `"inert"` + (default since 2026-10-10), `"propagate"`, or `"inert: no pathogen table"`, + and how many pathogen-derived nodes this solve held at baseline. A node is + held as a drug is under `inert`. - **`self_inhibitor_rule`** (additive): `"on"`, `"off"`, or `"inert: no containment table"`. The last means the solve did NOT run the default model, because the network came without a containment table. `/api/parse` returns `containment`; send it back with a POSTed network to keep the rule active. - **`cycle_rule`**, **`cycle_pools_solved`**, **`cycle_pools_multistep`**, **`cycle_carriers`**, **`cycle_carriers_held`** (additive, specs/039): diff --git a/specs/048-experimental-levers/research.md b/specs/048-experimental-levers/research.md index abf7dc5..ced5553 100644 --- a/specs/048-experimental-levers/research.md +++ b/specs/048-experimental-levers/research.md @@ -216,3 +216,19 @@ CREBBP knockout in DDX58/IFIH1. Nothing breaks on either axis. confirm the trace rather than test the idea blind. The gain rests on one curated SARS-CoV-2 complex in PIP3, and the experimental +7 is one perturbation. + +## Combination arm `combo` (registered before it runs) + +- **Build:** generator defaults at LNG `b85d5dc`, with `LNG_BIND_STOICH=1` + and `LNG_PATHOGEN_PROTEIN=1`. +- **Solver:** defaults at the commit that makes `DS_PATHOGEN_MODE=inert` the + default. +- **Compared with:** canonical (`20261009-2307_132d4b8`, `results/0f0a11e`). +- **Prediction:** the `pathp` effect alone, because `bind` moved no + prediction: + - experimental +7 (7/0, PDPK1 knockout); + - curator held-out +1 (CREBBP knockout in DDX58/IFIH1); + - tuning +14 (PIP3); + - no other prediction changes. +- **Any other change** is traced to a case before the build becomes + canonical. diff --git a/src/core/drugs.jl b/src/core/drugs.jl index 25aeff9..21eacd1 100644 --- a/src/core/drugs.jl +++ b/src/core/drugs.jl @@ -60,15 +60,20 @@ fold-change. The generator lists the nodes in `pathogens.csv` (a leaf with a species, none human; a complex with any such component; a set whose every member is one). - DS_PATHOGEN_MODE=propagate (default) — previous behaviour. - DS_PATHOGEN_MODE=inert — pathogen-derived nodes pinned at baseline, as + DS_PATHOGEN_MODE=inert (default since specs/048 amendment 2) — + pathogen-derived nodes pinned at baseline, as DS_DRUG_MODE=inert pins drugs. An explicit observation still wins. + DS_PATHOGEN_MODE=propagate — the previous behaviour. + +What the bundle lists is the generator's call. Since LNG_PATHOGEN_PROTEIN +(default on) only entities carrying a pathogen PROTEIN are listed: viral RNA +is the ligand DDX58/IFIH1 senses, and holding it cut 12 curator cases. """ const DS_VALID_PATHOGEN_MODES = Set(["propagate", "inert"]) function pathogen_mode()::String - value = get(ENV, "DS_PATHOGEN_MODE", "propagate") + value = get(ENV, "DS_PATHOGEN_MODE", "inert") if !(value in DS_VALID_PATHOGEN_MODES) throw(ArgumentError( "DS_PATHOGEN_MODE must be one of $(join(sort(collect(DS_VALID_PATHOGEN_MODES)), ", ")); got $(repr(value))")) diff --git a/test/test_drugs.jl b/test/test_drugs.jl index 89bca48..8057362 100644 --- a/test/test_drugs.jl +++ b/test/test_drugs.jl @@ -192,12 +192,18 @@ function psolve(net; mode, drug = nothing, x = 2.0, extra = Dict{String, Tuple{F end @testset "pathogens (specs/048)" begin - @testset "default is propagate, byte-identical to a network with no pathogen list" begin + @testset "default is inert; propagate is byte-identical to a network with no pathogen list" begin with_env("DS_PATHOGEN_MODE" => nothing) do - @test DS.pathogen_mode() == "propagate" + @test DS.pathogen_mode() == "inert" end - a = psolve(pfixture(Set(["R-C", "R-D"])); mode = nothing) - b = psolve(pfixture(nothing); mode = nothing) + d = psolve(pfixture(Set(["R-C", "R-D"])); mode = nothing) + @test d.node_activities == psolve(pfixture(Set(["R-C", "R-D"])); mode = "inert").node_activities + @test d.diagnostics["pathogen_rule"] == "inert" + n = psolve(pfixture(nothing); mode = nothing) + @test n.diagnostics["pathogen_rule"] == "inert: no pathogen table" + @test n.diagnostics["pathogens_held"] == 0 + a = psolve(pfixture(Set(["R-C", "R-D"])); mode = "propagate") + b = psolve(pfixture(nothing); mode = "propagate") @test a.node_activities == b.node_activities @test a.diagnostics["pathogen_rule"] == "propagate" @test a.diagnostics["pathogens_held"] == 0 @@ -221,7 +227,7 @@ end r = psolve(pfixture(nothing; drug_stids = Set(["R-C"])); mode = "inert") @test r.diagnostics["pathogen_rule"] == "inert: no pathogen table" @test fold(r, "C") ≈ 2.0 rtol = 1e-6 - r = psolve(pfixture(Set(["R-C"]); drug_stids = nothing); mode = nothing, drug = "inert") + r = psolve(pfixture(Set(["R-C"]); drug_stids = nothing); mode = "propagate", drug = "inert") @test r.diagnostics["drug_rule"] == "inert: no drug table" @test r.diagnostics["pathogens_held"] == 0 @test fold(r, "C") ≈ 2.0 rtol = 1e-6 From 9b59fa28b48a090c9f949492a5aecaee935f3154 Mon Sep 17 00:00:00 2001 From: Adam Wright <668955+adamjohnwright@users.noreply.github.com> Date: Sat, 10 Oct 2026 13:43:48 -0400 Subject: [PATCH 09/10] specs/048 combo arm: bind + pathogen-protein together meet the registered prediction exactly (exp +7, held-out +1, tuning +14, nothing else moves) Co-Authored-By: Claude Opus 5.5 (1M context) --- specs/048-experimental-levers/research.md | 28 +++++++++++++++++++++++ 1 file changed, 28 insertions(+) diff --git a/specs/048-experimental-levers/research.md b/specs/048-experimental-levers/research.md index ced5553..8072113 100644 --- a/specs/048-experimental-levers/research.md +++ b/specs/048-experimental-levers/research.md @@ -232,3 +232,31 @@ perturbation. - no other prediction changes. - **Any other change** is traced to a case before the build becomes canonical. + +### Result: prediction met exactly + +- **Build:** `20261010-1247_b85d5dc_combo`. + - It is complete, with 213,171 nodes and 516,528 edges. + - Per pathway, its networks match `bind` and its pathogen lists match + `pathp`. +- **Solver:** `2a3db74`, at code defaults. +- **Results:** `results/2a3db74/combo`. +- **Held:** 1,754 curator node-solves and 128 experimental. + +| axis | split | canonical → `combo` | net (fixed/broke) | McNemar p | +|---|---|---|---|---| +| experimental | all | 69.11% → 69.94% | **+7** (7/0) | 0.016 | +| curator | held-out | 88.99% → 89.00% | **+1** (1/0) | 1 | +| curator | tuning | 76.14% → 76.42% | +14 (14/0) | 0.00012 | +| curator | all | 86.24% → 86.31% | +15 (15/0) | 6.1e-05 | + +- **What changed:** exactly the 22 predicted cases, on two perturbations: + - PDPK1 knockout: 15 curator cases and 7 experimental; + - CREBBP knockout: 1 curator case. +- **Unchanged:** coverage and convergence (1,654 of 1,725 and 215 of 244 + solves). +- **Interaction:** the two changes do not interact. + +**Next.** The combination becomes canonical once LNG #110 is merged and +`current` is rebuilt from generator main. RESULTS.md is updated from that +build. From d3c33b220ebaab25cefa5ad4ab445abd22fa4149 Mon Sep 17 00:00:00 2001 From: Adam Wright <668955+adamjohnwright@users.noreply.github.com> Date: Sat, 10 Oct 2026 13:55:13 -0400 Subject: [PATCH 10/10] Review of #101: default-inert bench guard refuses only when listed pathogens match nothing; API round-trip test for pathogen_stids; numbers and wording fixed; old builds documented as needing propagate; review record in research.md Co-Authored-By: Claude Opus 5.5 (1M context) --- CLAUDE.md | 11 ++++++---- bench/benchmark_vs_mpbiopath.py | 11 +++++++++- bench/test_bench_failfast.py | 26 +++++++++++++++++++++++ docs/API.md | 3 +++ specs/048-experimental-levers/research.md | 22 ++++++++++++++++++- src/core/drugs.jl | 2 +- test/test_config_validation.jl | 8 +++++++ 7 files changed, 76 insertions(+), 7 deletions(-) diff --git a/CLAUDE.md b/CLAUDE.md index f1c97af..e4a2fbd 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -357,7 +357,10 @@ env vars only override for benchmark sweeps: `ReferenceGeneProduct`): a protein leaf, a complex with such a component, or a set whose members all are, e.g. SARS-CoV-2 N:M:PDPK1. - Viral RNAs are not listed. They are the ligand DDX58/IFIH1 senses, and - holding them cost 12 curator cases (arm `path`). + holding them broke 12 curator cases, net −11 (arm `path`). + - Builds from before 2026-10-10 have no `pathogens.csv`, so the bench refuses + them under the default; re-run one with `--server DS_PATHOGEN_MODE=propagate` + (the specs/047 MP-BioPath-network run included). - Against `propagate`: experimental +7 (all PDPK1 knockout), held-out +1, nothing broken. - The solve reports `pathogen_rule` and `pathogens_held`. A bundle without @@ -528,8 +531,8 @@ for the last feature that touched it rather than re-deriving from the code. five mechanism classes. `bench/analysis/experimental_gap.py` reproduces it. - `specs/048-experimental-levers/` — two levers from 047. - **Pathogen-derived nodes held at baseline: adopted**, as the protein-only - rule of amendment 2. The first rule also held viral RNA and lost 11 in - DDX58/IFIH1. + rule of amendment 2. The first rule also held viral RNA and broke 12 + cases in DDX58/IFIH1 (net −11). - **Homodimer binding fix (`LNG_BIND_STOICH`): adopted** on faithfulness; it moves no prediction. - Pools that share intermediates (the RAF knockout→UP class) are @@ -552,7 +555,7 @@ other seven execute code and print output. | file | assertions | note | |---|---|---| -| `test/test_config_validation.jl` | 212 | | +| `test/test_config_validation.jl` | 215 | | | `test/test_loop_elasticity.jl` | 173 | + 1 `@test_broken` | | `test/test_propagator_invariants.jl` | 121 | | | `test/test_loop_pool.jl` | 81 | | diff --git a/bench/benchmark_vs_mpbiopath.py b/bench/benchmark_vs_mpbiopath.py index f922ef6..e7a4cea 100644 --- a/bench/benchmark_vs_mpbiopath.py +++ b/bench/benchmark_vs_mpbiopath.py @@ -1076,6 +1076,9 @@ def run_pathway(pathway_id: str, pathway_name: str, gene_to_stids_cache=None, SOLVE_TALLY["drugs_held"] += int(ds_result.get("drugs_held") or 0) SOLVE_TALLY[f"pathogen_rule={ds_result.get('pathogen_rule', 'unknown')}"] += 1 SOLVE_TALLY["pathogens_held"] += int(ds_result.get("pathogens_held") or 0) + # how many listed stable ids this solve's network carried: a run whose + # pathways list none (most do) holds nothing under the default by design + SOLVE_TALLY["pathogens_listed"] += len(parsed.get("pathogen_stids") or []) SOLVE_TALLY[f"conserved_rule={ds_result.get('conserved_rule', 'unknown')}"] += 1 if ds_result.get("cycle_rule") == "balance: no pool table": raise SystemExit(f"{pathway_name}: DS_CYCLE_MODE=balance but the network has no pools.csv " @@ -1391,8 +1394,14 @@ def main(): raise SystemExit("DS_DRUG_MODE=inert but no solve held a drug node; the arm is the control.") prules = sorted(k.split("=", 1)[1] for k in SOLVE_TALLY if k.startswith("pathogen_rule=")) print(f"\nPathogens: rule {'/'.join(prules) or 'unknown'}, {SOLVE_TALLY['pathogens_held']} node-solves held (specs/048)") + # inert is the default since specs/048 amendment 2, and 83 of 92 pathways list + # no pathogen, so a subset run that holds nothing is expected. Refuse only when + # networks DID list pathogens and none was held: the listing matched nothing. if "inert" in prules and SOLVE_TALLY["pathogens_held"] == 0: - raise SystemExit("DS_PATHOGEN_MODE=inert but no solve held a pathogen node; the arm is the control.") + if SOLVE_TALLY["pathogens_listed"]: + raise SystemExit("DS_PATHOGEN_MODE=inert and the networks list pathogens, but no solve held one; " + "the listing matched no node.") + print(" (no pathway in this run lists a pathogen, so inert holds nothing: identical to propagate here)") yrules = sorted(k.split("=", 1)[1] for k in SOLVE_TALLY if k.startswith("cycle_rule=")) yc = sorted(k.split("=", 1)[1] for k in SOLVE_TALLY if k.startswith("cycle_carriers=")) print(f"\nCycles: rule {'/'.join(yrules) or 'unknown'}, {SOLVE_TALLY['cycle_pools_solved']} pool-solves, " diff --git a/bench/test_bench_failfast.py b/bench/test_bench_failfast.py index 33d94cd..023f6c3 100644 --- a/bench/test_bench_failfast.py +++ b/bench/test_bench_failfast.py @@ -50,3 +50,29 @@ def test_a_pathway_without_data_is_still_a_plain_skip(tmp_path, monkeypatch): outcomes = {"PathA": ok_result, "PathB": lambda n: {"status": "no_network", "name": n}} run_main(tmp_path, monkeypatch, outcomes) # returns normally: not a failure + + +def _pathogen_outcome(listed, held): + def outcome(name): + bench.SOLVE_TALLY["pathogen_rule=inert"] += 1 + bench.SOLVE_TALLY["pathogens_listed"] += listed + bench.SOLVE_TALLY["pathogens_held"] += held + return ok_result(name) + return outcome + + +def test_default_inert_on_pathways_listing_no_pathogen_is_not_refused(tmp_path, monkeypatch, capsys): + # specs/048: inert is the default and 83 of 92 pathways list no pathogen, so a + # subset run that holds nothing is the expected case, not "the arm is the control". + run_main(tmp_path, monkeypatch, {"PathA": _pathogen_outcome(0, 0)}) + assert "no pathway in this run lists a pathogen" in capsys.readouterr().out + + +def test_inert_refused_when_listed_pathogens_match_no_node(tmp_path, monkeypatch): + with pytest.raises(SystemExit) as e: + run_main(tmp_path, monkeypatch, {"PathA": _pathogen_outcome(5, 0)}) + assert "matched no node" in str(e.value.code) + + +def test_inert_holding_listed_pathogens_runs(tmp_path, monkeypatch): + run_main(tmp_path, monkeypatch, {"PathA": _pathogen_outcome(5, 3)}) diff --git a/docs/API.md b/docs/API.md index 72ec0c1..6a7a2f4 100644 --- a/docs/API.md +++ b/docs/API.md @@ -173,6 +173,9 @@ Response: (default since 2026-10-10), `"propagate"`, or `"inert: no pathogen table"`, and how many pathogen-derived nodes this solve held at baseline. A node is held as a drug is under `inert`. + The built-in sample network has no pathogen table, so a bodyless + `/api/parse` → `/api/solve` reports `"inert: no pathogen table"`; nothing + is wrong with that solve. - **`self_inhibitor_rule`** (additive): `"on"`, `"off"`, or `"inert: no containment table"`. The last means the solve did NOT run the default model, because the network came without a containment table. `/api/parse` returns `containment`; send it back with a POSTed network to keep the rule active. - **`cycle_rule`**, **`cycle_pools_solved`**, **`cycle_pools_multistep`**, **`cycle_carriers`**, **`cycle_carriers_held`** (additive, specs/039): diff --git a/specs/048-experimental-levers/research.md b/specs/048-experimental-levers/research.md index 8072113..1941569 100644 --- a/specs/048-experimental-levers/research.md +++ b/specs/048-experimental-levers/research.md @@ -251,7 +251,7 @@ perturbation. | curator | all | 86.24% → 86.31% | +15 (15/0) | 6.1e-05 | - **What changed:** exactly the 22 predicted cases, on two perturbations: - - PDPK1 knockout: 15 curator cases and 7 experimental; + - PDPK1 knockout: 14 curator cases and 7 experimental; - CREBBP knockout: 1 curator case. - **Unchanged:** coverage and convergence (1,654 of 1,725 and 215 of 244 solves). @@ -260,3 +260,23 @@ perturbation. **Next.** The combination becomes canonical once LNG #110 is merged and `current` is rebuilt from generator main. RESULTS.md is updated from that build. + +## Adversarial review (2026-10-10) + +Two independent reviewers looked at the diffs: DS #101 at `9b59fa2` and LNG +#110. Neither found a defect in propagation or in the measured networks. + +| finding | disposition | +|---|---| +| DS: the default `inert` refuses benchmarks on builds without `pathogens.csv`, which includes `current` | Documented in CLAUDE.md. **Merge order:** LNG #110, then a canonical rebuild, then DS #101. Old arms re-run with `DS_PATHOGEN_MODE=propagate`. | +| DS: a subset run under the default was refused ("the arm is the control") when its pathways list no pathogen | Fixed. The bench refuses only if the networks list pathogens and none is held. 3 bench tests added. | +| DS: research said PDPK1 changed 15 curator cases | Fixed: it is 14. | +| DS: the RNA cost was worded three ways | Now "broke 12, net −11" everywhere. | +| DS: nothing tested the API round trip of `pathogen_stids` | Added to `test_config_validation.jl` (+3). | +| DS: the silo-bridge test cannot fail if the rebuild drops the field | **Not fixed.** The code is correct today. Making the test fail would need a fixture whose pins are resolved after the rebuild. | +| DS: the sample network reports `inert: no pathogen table` | Documented in API.md. | +| LNG: a viral Polymer (HIV-1 Rev-multimer) was not a pathogen under the protein rule | Fixed. `repeatedUnit` is read as a component. Recomputed on the combo build, it lists no new entity in any of the 92 pathways (`analysis/048/polymer_check.py`), so the measured arms stand. | +| LNG: the signature cache was not keyed on `LNG_BIND_STOICH` | Fixed, with a test. | +| LNG: the docstring said viral RNAs have no reference entity | Fixed: most are EWAS on a `ReferenceRNASequence`, and the class allow-list excludes them. | +| LNG: `_leaf_multiset` was tested at one level only | Nested-stoichiometry test added. | +| LNG: the pathogen cache depends on call order beyond depth 10; catalysts outside prefetch get no multiset | **Not fixed.** Real nesting tops out around 5, and the catalyst case is not a regression. | diff --git a/src/core/drugs.jl b/src/core/drugs.jl index 21eacd1..e768e48 100644 --- a/src/core/drugs.jl +++ b/src/core/drugs.jl @@ -68,7 +68,7 @@ member is one). What the bundle lists is the generator's call. Since LNG_PATHOGEN_PROTEIN (default on) only entities carrying a pathogen PROTEIN are listed: viral RNA -is the ligand DDX58/IFIH1 senses, and holding it cut 12 curator cases. +is the ligand DDX58/IFIH1 senses, and holding it broke 12 curator cases (net −11). """ const DS_VALID_PATHOGEN_MODES = Set(["propagate", "inert"]) diff --git a/test/test_config_validation.jl b/test/test_config_validation.jl index a7add3b..59e1033 100644 --- a/test/test_config_validation.jl +++ b/test/test_config_validation.jl @@ -675,6 +675,14 @@ end net2 = DeltaSignal.reaction_network_from_json(JSON3.read(with_cof)) @test net2.cofactor_stids == Set(["R-ALL-29372", "R-ALL-113582"]) @test length(net2.nodes) == 2 + + # specs/048: the pathogen list must survive too, or the default + # DS_PATHOGEN_MODE=inert silently reports "inert: no pathogen table". + with_path = replace(payload, "\"edges\":" => "\"pathogen_stids\": [\"R-HSA-2\"], \"edges\":") + net3 = DeltaSignal.reaction_network_from_json(JSON3.read(with_path)) + @test net3.pathogen_stids == Set(["R-HSA-2"]) + @test net.pathogen_stids === nothing + @test DeltaSignal.pathogen_uuids(net3) == Set(["B"]) end @testset "edge sign and logic values fail loudly" begin