Real-API solo susceptibility run: MIMIC-CXR text at n=35 then n=600

[felipeocampoos]

Context

Tracking sub-task of #316 (Solo shortcut susceptibility + noise floor, MIMIC-CXR text), itself part of #296. The MCQ template/distractor design was settled in #330.

What this covers

Real-API runs of experiments/medqa/reproduce.py (reused as-is, per the "do not rewrite the method, replicate it" instruction on #316) against the new benchmaxxing/datasets/mimic_cxr_text.py manifest, on real MIMIC-CXR report text via the real Gemini API:

python -m experiments.medqa.reproduce --manifest experiments/mimic_cxr_text/mimic_cxr_text_manifest.csv \
    --out experiments/mimic_cxr_text/results --stage solo --solo-n <N>

Per @sebasmos's guidance: run at n=35 first, then scale to n=600 if time allows (CheXpert/MIMIC has enough eligible studies after the #330 filters to support both).

Definition of done

• n=35 real-API run committed under experiments/mimic_cxr_text/results/ (solo flip rate + noise floor per model/cue, in the same JSON/JSONL shape as experiments/medqa/results/)
• n=600 real-API run committed likewise, if time allows
• Numbers read off cleanly into the cross-dataset comparison table (MIMIC-CXR text (reports) standard-battery replication (tracking) #296)

Part of #316. Results + code in PR #336.

[sebasmos]

For sizing reference, MedQA's own real-API counts per experiment: solo susceptibility n=100, cascade/referee n=40 (15 gamed), break-it hard-case tests n=85. n=600 here is in that same range.

The n=35 first makes sense for a different reason than sizing: mimic_cxr_text.py is a brand-new adapter never run against the real API, so 35 is a cheap smoke test to catch a broken parser/manifest before spending ~600 calls on it, not because the 35-to-600 nesting itself needs re-validating (that was already proven for MIMIC images via #263).

[felipeocampoos]

Both runs done, real Gemini API, results in PR #336:

n=35 (smoke):

{"gemini-2.5-flash": {"flip": 0.095, "noise_floor": 0.133},
 "gemini-2.5-flash-lite": {"flip": 0.124, "noise_floor": 0.0}}

n=600:

{"gemini-2.5-flash": {"flip": 0.079, "noise_floor": 0.0},
 "gemini-2.5-flash-lite": {"flip": 0.124, "noise_floor": 0.0}}

n=35's flash noise floor (0.133 > flip rate) was a small-sample artifact from the fixed
15-case noise-floor control; resolved cleanly at n=600 (0.0 for both models). Flip rates
stable across both sample sizes. Full detail in experiments/mimic_cxr_text/RESULTS.md
on the PR.

[felipeocampoos]

Both real-API runs (n=35, n=600) are committed under experiments/mimic_cxr_text/results/ — see PR #336 (approved by @sebasmos, pending merge). Closing as done; the deliverable exists and is reviewed, independent of merge timing.