docs(n50): a clean N50' cannot be sourced from repo data — deferred to human-led curation#106
Merged
Merged
Conversation
…— deferred to human-led curation A 3-agent investigation (spec/requirements, IK14 tooling, candidate-pool feasibility) scoped the clean N50' re-curation and answered the make-or-break question first: a clean N=50 is not sourceable from the existing reference data. Feasibility funnel (clinical_pk.json): 331 -> 177 with Cmax -> 16 after removing the 107-holdout + 76-train (name AND InChIKey-14) -> 2 nominal clean -> 0 valid (guanfacine ER = holdout HCl-salt IK14 leak; lanthanum carbonate = inorganic combo). OATP1B1 non-statin substrates in the clean pool: 0 of >=3 required. N=50 infeasible by -50; the clinical reference IS the training source (167/331 names in holdout+train) and DrugBank covers 14,154 IK14 blocks. A real N50' needs ~50 fresh novel molecules with human-verified primary-source Cmax (no back-calc), CID-verified SMILES, >=3 non-statin OATP1B1 substrates, oral-majority. Integrity constraint: an agent must not fabricate primary values for a never-touch instrument (undetectable by the IK14 gate, worse than the 2026Q2 contamination). E4 (DrugBank-absent) could relax to hard-corpora-only if the benchmark commits to public-clone, but the repo pool is still ~2-5. Deferred to a separately-authorized human curation cycle; tooling (#96) + spec are ready. Adds the feasibility doc, a DE-53 status line, an experiment-log entry, and flags a salt-canonicalization gap in the IK14 gate. Headline 2.743 untouched.
jam-sudo
enabled auto-merge (squash)
July 8, 2026 01:25
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What
Scopes the clean N50' secondary-holdout re-curation (the unbiased-generalization instrument the cherry-picking caveat calls for) and records the make-or-break finding: a clean N=50 is not sourceable from existing repository data.
Feasibility funnel (
clinical_pk.json)The 2 nominal survivors both fail: guanfacine ER = holdout HCl-salt IK14 leak (free-base is in holdout; the counterion defeated the gate), lanthanum carbonate = inorganic combo. N=50 infeasible by −50; even N=1 isn't achievable — the clinical reference is the training source (167/331 names in holdout+train), and DrugBank covers 14,154 IK14 blocks (2026Q2 was 47/50 in DrugBank).
Why deferred to human-led curation
A real N50' needs ~50 fresh novel molecules with primary-source Cmax (no back-calc), CID-verified SMILES, ≥3 non-statin OATP1B1 substrates, oral-majority. Integrity constraint: an agent must NOT generate primary-source Cmax/SMILES/DOI for a never-touch instrument — a fabricated value is undetectable by the IK14 gate (structure membership, not value provenance) and would be worse than the 2026Q2 contamination.
E4 / public-clone reframe (a yield lever, not a rescue)
The 2.743 headline is public-clone (DrugBank hidden → no enrichment leak), so E4 (DrugBank-absent) could relax to hard-corpora-only if the N50 benchmark commits to public-clone. Even so the repo pool is ~2–5, not 50.
Changes
docs/research/n50_prime_feasibility_2026-07-07.md(new): the assessment.dead-ends.md: DE-53 status line.experiment-log.md: 2026-07-07 (cont.) entry.build_n50_exclusion.py's IK14 gate (false-negative-only).(Local-only spec
2026-04-22-n50-secondary-holdout-design.mdupdated to match; gitignored.)Headline 2.743 untouched. Assessment/docs only — no code, data, or curation.