MedMCQA standard-battery replication (tracking)

[duckyquang]

Tracking issue for the MedMCQA standard-battery replication. Each item mirrors an experiment from the finished MedQA/NIH template so results are apples-to-apples across datasets (see the workflow: replicate the standard battery on a new dataset).

Dataset adapter is already implemented (benchmaxxing/datasets/medmcqa.py, #112). These are replication, not new features: run the existing experiments on a MedMCQA manifest through the Gemini API, and write results under experiments/medmcqa/results/ in the same JSON/JSONL format as experiments/medqa/results/.

Foundational (run first)

• Solo shortcut susceptibility + noise floor (MedMCQA) #267 Solo shortcut susceptibility + noise floor (MedMCQA)
• Cascade baseline: bare and anchored seed (MedMCQA) #268 Cascade baseline: bare and anchored seed (MedMCQA)

Seed shape

• Plausible vs implausible planted distractor (MedMCQA) #269 Plausible vs implausible planted distractor (MedMCQA)
• Rationale validity: bare vs valid-but-wrong vs named-fallacy seed (MedMCQA) #270 Rationale validity: bare vs valid-but-wrong vs named-fallacy seed (MedMCQA)
• Seed confidence: hedged vs confident wrong peer (MedMCQA) #271 Seed confidence: hedged vs confident wrong peer (MedMCQA)
• Attributed-tier identity of the seed (MedMCQA) #272 Attributed-tier identity of the seed (MedMCQA)
• Cue dose-response: text magnitude sweep (MedMCQA) #273 Cue dose-response: text magnitude sweep (MedMCQA)
• Text cue types: primacy/position and negation/qualifier (MedMCQA) #274 Text cue types: primacy/position and negation/qualifier (MedMCQA)

Social pressure / committee structure

• Majority-pressure (Asch) variant (MedMCQA) #275 Majority-pressure (Asch) variant (MedMCQA)
• Committee-size sweep: dilution and proportional pressure (MedMCQA) #276 Committee-size sweep: dilution and proportional pressure (MedMCQA)
• Dissenter / unanimity break (MedMCQA) #277 Dissenter / unanimity break (MedMCQA)
• Deliberation framing crossed with the anchored seed (MedMCQA) #278 Deliberation framing crossed with the anchored seed (MedMCQA)
• Super-additivity 2x2: system flag AND anchored senior peer (MedMCQA) #279 Super-additivity 2x2: system flag AND anchored senior peer (MedMCQA)

Hierarchy / orchestrator

• Hierarchy dominance via order-permutation committees (MedMCQA) #280 Hierarchy dominance via order-permutation committees (MedMCQA)
• Wrong orchestrator vs wrong peer (MedMCQA) #281 Wrong orchestrator vs wrong peer (MedMCQA)
• Authority gradient ladder (MedMCQA) #282 Authority gradient ladder (MedMCQA)
• Leader-as-auditor remediation prompt (MedMCQA) #283 Leader-as-auditor remediation prompt (MedMCQA)

Controls & robustness

• True-peer negative control: plant the CORRECT answer (MedMCQA) #284 True-peer negative control: plant the CORRECT answer (MedMCQA)
• Test-awareness manipulation on the cascade (MedMCQA) #285 Test-awareness manipulation on the cascade (MedMCQA)
• Temperature sensitivity + run-to-run reliability (MedMCQA) #286 Temperature sensitivity + run-to-run reliability (MedMCQA)
• Onset + contagion/deference battery over transcripts (MedMCQA) #287 Onset + contagion/deference battery over transcripts (MedMCQA)

Break-it arms

• Break-it A: contaminated shared context / system pre-screen flag (MedMCQA) #288 Break-it A: contaminated shared context / system pre-screen flag (MedMCQA)
• Break-it C: case-anchored plausible seed vs generic (MedMCQA) #289 Break-it C: case-anchored plausible seed vs generic (MedMCQA)
• Break-it D: blind-metric decoy rewards the longest option (MedMCQA) #290 Break-it D: blind-metric decoy rewards the longest option (MedMCQA)

Related: #129 (MedQA vs MedMCQA cross-dataset consistency, @Agastya191) is adjacent - coordinate to avoid duplicate cascade runs.

Referee (was missing; the named contribution)

• Referee detection: naive gate vs targeted vs deployable (MedMCQA) #322 Referee detection: naive gate vs targeted vs deployable

[sebasmos]

Heads up: the solo-cue cell is already shown consistent with MedQA via #266. The full battery (#267-#290) hasn't started yet though, that's the sprint priority right now over more infra PRs, deadline's the 29th.