Skip to content

docs(dead-ends): DE-57 — CLint epistemic-uncertainty gate fails (uncertainty-aware meta foreclosed)#107

Merged
jam-sudo merged 1 commit into
mainfrom
research/de57-clint-uncertainty-gate
Jul 9, 2026
Merged

docs(dead-ends): DE-57 — CLint epistemic-uncertainty gate fails (uncertainty-aware meta foreclosed)#107
jam-sudo merged 1 commit into
mainfrom
research/de57-clint-uncertainty-gate

Conversation

@jam-sudo

@jam-sudo jam-sudo commented Jul 8, 2026

Copy link
Copy Markdown
Owner

Summary

Pressure-tests the last diagnosis.md §9 CLint-floor candidate — calibrated-CLint-uncertainty → uncertainty-aware meta — against the §4 decorrelation gate. It is the one candidate that could structurally sidestep error-cancellation, because it is a per-drug weighting signal (down-weight the engine track where CLint is unreliable), not a point-estimate change. Verdict: FAIL → DE-57. Headline 2.743 untouched (gate pre-check, nothing integrated).

Method

Constructs a genuine per-drug CLint epistemic uncertainty two independent ways and correlates each with per-drug Cmax residuals (Spearman + bootstrap 95% CI, seed 20260422, N=107 from the canonical cache):

  • U_boot — SD of log10(CLint) across a K=30 XGBoost bootstrap ensemble on the 1,213-compound hepatocyte training set (non-degenerate, range [0.069, 0.412]).
  • U_ad — 1 − max Tanimoto(Morgan) to that training set (applicability-domain distance).

Gate: PASS iff any |ρ| ≥ 0.30 with CI excluding 0.

Result

signal vs e_eng (engine err) vs e_eng−e_ml (actionable) vs r_meta
U_boot −0.006 [−0.20, +0.18] +0.119 [−0.07, +0.30] −0.255 [−0.42, −0.07]
U_ad +0.145 [−0.06, +0.34] +0.121 [−0.07, +0.31] +0.032 [−0.16, +0.23]
  • Core null: CLint epistemic uncertainty does not predict engine error, though the engine is the CLint-fed track.
  • Actionable null: both independent proxies give ρ≈+0.12 with CI∋0 on e_eng−e_ml — no reweighting signal for the meta.
  • The one CI-significant term (ρ(U_boot, r_meta)=−0.255) is sub-threshold and wrong-signed (higher CLint uncertainty ⇒ lower meta error).
  • Confound-clean: ρ(U_boot, obs-magnitude)=+0.002.

Why it fails (§10)

The binding wall is bioavailability-F blindness, shared across all four tracks and orthogonal to CLint. Cmax error is F-driven, so no CLint-derived signal can discriminate it — a perfect CLint-uncertainty router would have nothing to route on. Per §4 the gate precedes integration → the uncertainty-aware meta is not built. Distinct from DE-55 (which tested the propagated MC-output uncertainty, ρ=0.039; the shipped CLint CV is the fixed constant _CLINT_CV=1.0).

Changes

  • docs/research/dead-ends.md — DE-57 entry
  • docs/research/diagnosis.md — §9 open-follow-up marked closed
  • scripts/clint_uncertainty_gate.py — reproducible gate (PYTHONPATH=src python scripts/clint_uncertainty_gate.py, deterministic, repo-data only)
  • data/validation/clint_uncertainty_gate_2026-07-08.json — result artifact

Sanitization guard passes (711 tracked files); ruff clean.

…rtainty-aware meta foreclosed

The last diagnosis.md §9 CLint-floor candidate that could structurally sidestep
error-cancellation (a per-drug weighting signal, not a point-estimate change).
Gate-tested two independent per-drug CLint epistemic-uncertainty constructions
(K=30 XGBoost bootstrap-ensemble SD; AD max-Tanimoto distance) against per-drug
Cmax residuals (Spearman + bootstrap 95% CI, seed 20260422, N=107):

  rho(U_boot, e_eng)        = -0.006 [-0.20, +0.18]  (engine is the CLint-fed track)
  rho(U, e_eng - e_ml)      ~ +0.12  [CI includes 0] for BOTH proxies (actionable null)

The one CI-significant term rho(U_boot, r_meta)=-0.255 is sub-threshold and
wrong-signed. Mechanistically consistent with §10: Cmax error is bioavailability-F
driven and orthogonal to CLint, so no CLint-derived signal can gate the meta. Per §4
the gate precedes integration -> not built. Distinct from DE-55 (which tested the
propagated MC-output uncertainty; the shipped CLint CV is the fixed _CLINT_CV=1.0).

- scripts/clint_uncertainty_gate.py: reproducible gate (PYTHONPATH=src)
- data/validation/clint_uncertainty_gate_2026-07-08.json: result artifact
- diagnosis.md §9: uncertainty-aware-meta follow-up marked closed

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: f24d087440

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

e_eng = np.abs(np.log10(eng / obs + eps))
e_ml = np.abs(np.log10(ml / obs + eps))
diff = e_eng - e_ml
mag = np.abs(np.log10(obs + eps)) # confound: obs magnitude

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Use a monotone observed-magnitude confound

For this holdout cache, observed Cmax spans 0.00040072–32.9 mg/L, with values on both sides of 1 mg/L, so abs(log10(obs)) ranks both very small and very large concentrations as “large” and treats values near 1 as small. That is distance from 1 mg/L rather than observed magnitude, so the reported obs_mag correlation can look clean even if uncertainty is confounded with high-vs-low exposure; use a monotone quantity such as log10(obs) or raw obs ranks before claiming the DE-57 gate is magnitude-confound-clean.

Useful? React with 👍 / 👎.

@jam-sudo
jam-sudo merged commit 7fd9e4f into main Jul 9, 2026
2 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant