docs(dead-ends): DE-57 — CLint epistemic-uncertainty gate fails (uncertainty-aware meta foreclosed)#107
Conversation
…rtainty-aware meta foreclosed The last diagnosis.md §9 CLint-floor candidate that could structurally sidestep error-cancellation (a per-drug weighting signal, not a point-estimate change). Gate-tested two independent per-drug CLint epistemic-uncertainty constructions (K=30 XGBoost bootstrap-ensemble SD; AD max-Tanimoto distance) against per-drug Cmax residuals (Spearman + bootstrap 95% CI, seed 20260422, N=107): rho(U_boot, e_eng) = -0.006 [-0.20, +0.18] (engine is the CLint-fed track) rho(U, e_eng - e_ml) ~ +0.12 [CI includes 0] for BOTH proxies (actionable null) The one CI-significant term rho(U_boot, r_meta)=-0.255 is sub-threshold and wrong-signed. Mechanistically consistent with §10: Cmax error is bioavailability-F driven and orthogonal to CLint, so no CLint-derived signal can gate the meta. Per §4 the gate precedes integration -> not built. Distinct from DE-55 (which tested the propagated MC-output uncertainty; the shipped CLint CV is the fixed _CLINT_CV=1.0). - scripts/clint_uncertainty_gate.py: reproducible gate (PYTHONPATH=src) - data/validation/clint_uncertainty_gate_2026-07-08.json: result artifact - diagnosis.md §9: uncertainty-aware-meta follow-up marked closed
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: f24d087440
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
| e_eng = np.abs(np.log10(eng / obs + eps)) | ||
| e_ml = np.abs(np.log10(ml / obs + eps)) | ||
| diff = e_eng - e_ml | ||
| mag = np.abs(np.log10(obs + eps)) # confound: obs magnitude |
There was a problem hiding this comment.
Use a monotone observed-magnitude confound
For this holdout cache, observed Cmax spans 0.00040072–32.9 mg/L, with values on both sides of 1 mg/L, so abs(log10(obs)) ranks both very small and very large concentrations as “large” and treats values near 1 as small. That is distance from 1 mg/L rather than observed magnitude, so the reported obs_mag correlation can look clean even if uncertainty is confounded with high-vs-low exposure; use a monotone quantity such as log10(obs) or raw obs ranks before claiming the DE-57 gate is magnitude-confound-clean.
Useful? React with 👍 / 👎.
Summary
Pressure-tests the last diagnosis.md §9 CLint-floor candidate — calibrated-CLint-uncertainty → uncertainty-aware meta — against the §4 decorrelation gate. It is the one candidate that could structurally sidestep error-cancellation, because it is a per-drug weighting signal (down-weight the engine track where CLint is unreliable), not a point-estimate change. Verdict: FAIL → DE-57. Headline 2.743 untouched (gate pre-check, nothing integrated).
Method
Constructs a genuine per-drug CLint epistemic uncertainty two independent ways and correlates each with per-drug Cmax residuals (Spearman + bootstrap 95% CI, seed 20260422, N=107 from the canonical cache):
U_boot— SD of log10(CLint) across a K=30 XGBoost bootstrap ensemble on the 1,213-compound hepatocyte training set (non-degenerate, range [0.069, 0.412]).U_ad— 1 − max Tanimoto(Morgan) to that training set (applicability-domain distance).Gate: PASS iff any |ρ| ≥ 0.30 with CI excluding 0.
Result
e_eng(engine err)e_eng−e_ml(actionable)r_metae_eng−e_ml— no reweighting signal for the meta.Why it fails (§10)
The binding wall is bioavailability-F blindness, shared across all four tracks and orthogonal to CLint. Cmax error is F-driven, so no CLint-derived signal can discriminate it — a perfect CLint-uncertainty router would have nothing to route on. Per §4 the gate precedes integration → the uncertainty-aware meta is not built. Distinct from DE-55 (which tested the propagated MC-output uncertainty, ρ=0.039; the shipped CLint CV is the fixed constant
_CLINT_CV=1.0).Changes
docs/research/dead-ends.md— DE-57 entrydocs/research/diagnosis.md— §9 open-follow-up marked closedscripts/clint_uncertainty_gate.py— reproducible gate (PYTHONPATH=src python scripts/clint_uncertainty_gate.py, deterministic, repo-data only)data/validation/clint_uncertainty_gate_2026-07-08.json— result artifactSanitization guard passes (711 tracked files); ruff clean.