fix(predict): disable uninformative RBP model behind an honest 1.0 default#101
Merged
Conversation
…fault The RBP path carried a compound bug. xgboost_rbp.json's target is a raw blood:plasma ratio (per its meta.json), but _predict_rbp read the output as log10(RBP) and applied 10**, inflating every prediction (caffeine 0.72 -> 5.3) past the [0.5, 3.0] clip ceiling. An asymmetric reset (abs(rbp-1.0) > 0.5) whose low branch was unreachable then snapped the clipped 3.0 down to 1.0. Net effect: all 107 holdout drugs resolved to exactly 1.0 regardless of structure. The model is uninformative anyway (scaffold-CV R2=-0.08, worse than the mean), and its meta.json documents the intended behaviour outright: default to 1.0. Make that explicit instead of incidental: _predict_rbp() returns a constant Distribution(1.0, cv=_RBP_CV). Output is bit-identical to the old buggy path on every holdout drug (verified via the real predict path), so no benchmark re-pin is required. The model file is retained for a future retrain. Removes both bugs (the 10** mis-transform and the dead-branch reset) and the misleading log_rbp naming. Adds guards that _predict_rbp() is a constant 1.0 and that RBC partitioners (chloroquine, tacrolimus) are not distinguished.
jam-sudo
enabled auto-merge (squash)
July 5, 2026 01:57
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What
The RBP prediction path carried a compound bug that this PR replaces with an explicit, honest default.
xgboost_rbp.json's target is a raw blood:plasma ratio (per itsmeta.json), but_predict_rbpread the model output aslog10(RBP)and applied10**— inflating every prediction (e.g. caffeine 0.72 → 5.3) past the[0.5, 3.0]clip ceiling. An asymmetric resetabs(rbp - 1.0) > 0.5— whose low branch is unreachable (clip floor 0.5 sits exactly on the threshold) — then snapped the clipped 3.0 down to 1.0.Net effect: all 107 holdout drugs resolved to exactly 1.0 regardless of structure (verified empirically: KEPT=0/107).
Why this is the right fix
The model is uninformative — scaffold-CV R²=−0.08, i.e. worse than predicting the population mean — and its own
meta.jsondocuments the intended behaviour: "Pipeline defaults to RBP=1.0." Activating the model (fixing only the10**) would inject known-worse-than-mean noise into all 107 drugs and require a benchmark re-pin, against the correctness-first principle. Instead,_predict_rbp()now returns a constantDistribution(1.0, cv=_RBP_CV), making the documented default explicit rather than incidental.Headline impact: none (bit-identical)
Both the old buggy path and the new constant yield
Distribution(mean=1.0, cv=0.3)for every holdout drug — confirmed via the realpredict_admepath on a diverse sample (sirolimus, digoxin, morphine, quinine, tamoxifen, abacavir, famotidine). No canonical regen / re-pin required.Changes
_predict_rbp→ constantDistribution(1.0, cv=_RBP_CV); removes the10**mis-transform, the dead-branch reset, and the misleadinglog_rbpnaming. Docstring records the history.rbp = _predict_rbp()).xgboost_rbp.jsonretained for a future retrain._predict_rbp()is a constant 1.0 with_RBP_CV; RBC partitioners (chloroquine, tacrolimus) are not distinguished from aspirin/benzene.Tests
tests/unit/test_adme_ivive.py22 passed; broad downstream sweep (predict/engine-flux/pipeline/model-registry) 134 passed.