feat/social-games-elo-eval

[Keyu-He]

Closes #

📑 Description

✅ Checks

• My pull request adheres to the code style of this project
• My code requires changes to the documentation
• I have updated the documentation as required
• All the tests have passed
• Branch name follows type/descript (e.g. feature/add-llm-agents)
• Ready for code review

ℹ Additional Information

[XuhuiZhou]

Thanks for the substantial work here. I took a pass over the current head (a5d7ecadb82e668d5393bbc1fbb8475ba8bd7704).

For this to be mergeable, I would like it reduced to a minimal, backward-compatible PR:

• Rebase onto current main and preserve its existing public APIs, defaults, and recipient/private-action semantics.
• Do not change behavior for existing callers unless the change is explicitly opt-in.
• Keep this PR to the smallest shared social-game engine change needed, with focused compatibility tests.
• Move the 21 game implementations, tournament/Elo tooling, reflection pipeline, generated results, and notebooks into separate follow-up PRs.

There are also several concrete blockers in the current branch:

1. It does not merge cleanly with current main. There are conflicts in .gitignore, llm_agent.py, logs.py, parallel.py, social_game.py (add/add), and generate.py. Some are semantic rather than mechanical, so resolving them by taking the PR side could regress current functionality.

2. The repository's strict type-check fails. Running uv run mypy --strict . reports 23 errors across eight changed or added files, including the recursive fallback in generate.py:719.

3. Unsupported structured output can silently turn an agent into a no-op. The capability check may raise response_format is not supported (lines 311-319), but the fallback only retries errors containing response_schema. Other failures return action_type=none (lines 715-739), which can silently bias model comparisons.

4. The Werewolf teammate prompt is unreachable. The later check compares a configured goal sentence to the literal string Werewolf (lines 480-516), so partner identities are never added as intended.

5. The Elo output is not reproducible and can silently omit data. Logs are shuffled without a fixed seed while ELO_EPOCHS = 1 (lines 362-364), and broad except Exception: continue paths silently discard malformed logs (lines 367-485).

6. The reset API ignores its caller. include_background_observations is accepted but the parent call always receives False (lines 275-292). Current main already passes this through correctly.

Ruff passes. The broad pytest run was inconclusive locally because parts of the existing suite require an external model endpoint and a specific Redis setup. I also did not find new tests for the shared engine, game state machines, or Elo calculation.

My preferred next step is therefore: a small rebased engine PR with backward-compatibility tests first, followed by separately reviewable game and evaluation PRs.