[michalmaj]

Summary

Targeted hotfix for the three problems found by the final quality gate on final_analytics_epilogue:

1. Deep-dive primary and variant both had hero_win_rate=1.0 (couldn't distinguish a hardcoded value).
2. most_one_sided_matchup was informationally empty (every one of the 9 matchups ties at abs(rate-0.5)==0.5 in this frozen combat model — confirmed mathematically and empirically), and most_balanced_class was a 3-way tie in the recorded primary history, broken only by variant sampling weights, not any real property of a hero class.
3. hardest_monster, rank_monsters, and unfavorable_matchups are mathematically invariant to any run-count reweighting in this frozen domain — the primary/variant fixture pair alone can't catch a hardcoded answer for them.

Per the instructions, this is not solved by expanding AST/dataflow analysis — it's solved with behavioral probes on genuinely different real inputs.

Changes

Commit 1 — fixture signal:

• scripts/generate_fixtures.py: deep-dive variant changed from rogue_vs_Goblin (also hero_win_rate=1.0) to warrior_vs_Dragon (hero_win_rate=0.0 — a genuinely opposite outcome, not just a different hero-wins matchup). Regenerated via the same real BattleService orchestration; verified byte-for-byte deterministic across two clean regenerations.

Commit 2 — behaviorally robust conclusions:

• task.py: balance_conclusions() drops most_balanced_class and most_one_sided_matchup; adds longest_matchup (matchup with highest avg avg_rounds — a real, continuously-varying signal) and one_sided_matchup_count (count of matchups with exactly 0.0 or 1.0 win rate — turns the deterministic-win-rate finding into the conclusion itself instead of hiding it behind a forced ranking). Docstrings make clear these functions must work generically on whatever df subset they're given.
• check.py:
  • load_battle_arrays's gold array values are now verified against the fixture (previously only shape/type were checked — a wrong gold array silently passed).
  • New behavioral subset probes: balance_conclusions, rank_monsters, and unfavorable_matchups are additionally run on controlled slices of the real primary tournament_history.csv (e.g. Orc+Dragon-only rows, warrior-only rows, mage-minus-Orc rows) — every row is genuine, just a subset. A function hardcoded to the full 9-matchup fixture fails these even though it passes the primary/variant pair.
  • Existing technique gates (NumPy reductions/percentile-or-quantile/mask/axis, pandas groupby) are unchanged — only regression-tested.
• README.md: reframes win_rate_by_class as "observed in this recorded history," not inherent class balance; questions section replaces the two removed conclusions and adds a short note on why ("when a planned metric turns out not to distinguish any of your cases, change the question"), without revealing the actual answers.

Test plan

• Scaffold → FAIL
• Reference solution → PASS (primary + variant + all subset probes)
• Alternative-valid B (np.asarray/np.quantile/np.count_nonzero/np.stack/.query/named .agg/sort_values().index[0] instead of idxmin/idxmax) → PASS
• Fake A (hero_win_rate=1.0 hardcoded) → FAIL on the new losing variant
• Fake B (gold array wrong) → FAIL on the new loader value check
• Fake C (hardest_monster hardcoded) → PASS primary/variant, FAIL on Orc-only subset
• Fake D (rank_monsters hardcoded + ceremonial groupby) → FAIL on Orc+Dragon subset
• Fake E (unfavorable_matchups hardcoded + ceremonial groupby) → FAIL on warrior-slice subset
• Fake F (longest_matchup hardcoded) → FAIL on no-warrior_vs_Dragon subset
• Fake G (one_sided_matchup_count=9 hardcoded) → FAIL on warrior-only subset
• Fake H (stale build_final_report) → FAIL on variant
• Fake I (ceremonial NumPy) → FAIL technique gate (unchanged)
• Fake J (partial dataset) → FAIL on values
• tools/author_check.py → 750 checks passed, 0 errors
• tools/course_status.py → epilogue still tracked correctly
• Data Recall Checkpoint smoke test, Levels 1-7 untouched (git diff --stat against merge base empty)
• Scaffold restored byte-for-byte after every test run; no leftover .progress/cache in the working tree

🤖 Generated with Claude Code