feat(coin-gym): LOCAL COIN Gym harness scaffold — baseline-vs-team + overfit gate (#2713 Phase 4)

[rysweet]

Summary

Phase 4 of the LOCAL COIN Gym goal (issue #2713): a Rust coin_gym module + coin-gym CLI that run the COIN benchmark shape locally, score reach/precision against the published leaderboard, and A/B a single-model baseline vs. a multi-agent team — mirroring skwaq's failure-analysis + overfitting-reviewer gating (design: docs/research/coin-benchmark-and-skwaq-study.md, Part 3, landed via #2712).

The whole pipeline runs offline against a mock oracle, so it is fully exercised without a VM. Real coin evaluate grading (Docker/VM) is Phase 3; the live self-improvement verify/rollback loop is Phase 5 — both remain unchecked follow-ups on #2713.

Why Rust (not a Python coin-gym/ package)

CI enforces a Rust-only policy (#2155, scripts/check-rust-only-gate.sh — the first CI gate), and the design doc itself calls the harness a "Simard crate"; the repo already ships simard-gym/simard-rust-gym. COIN's own coin evaluate (Python/uv/Docker) stays an external oracle the harness delegates to via a mockable executor — never re-implemented (exactly the executor contract in the task/design).

Components (design Part 3.3)

• Target loader — pinned + held-out fresh slice from a JSON snapshot manifest (TargetSource trait; fixture + in-memory impls; bundled sample).
• Agent runner — two strategies behind one AgentStrategy interface: BaselineStrategy (single model) and TeamStrategy (reacher/skeptic/synthesizer with a threshold_hint submit-or-abstain gate). Reasoning abstracted behind a mockable Reasoner.
• Harness executor — HarnessExecutor trait; CoinEvaluateExecutor builds the coin evaluate delegation argv (Phase-3 gated, unit-tested) + MockHarnessExecutor deterministic test double. The oracle is never re-implemented.
• Scorer — reach rate + precision, family split (frontier / non-trivial reachable), R/W/A/T/N/E histogram.
• Leaderboard comparator — published targeted-track numbers vs. local; flags material deviation (>10 pts) as a harness/config bug.
• Offline failure-analyst + overfitting-reviewer gate — the anti-overfit crux: turns unreached targets into general tactics and rejects any that memorize an input or key off a specific target/project/locator. (Live verify/rollback = Phase 5.)
• Profiles — isolated per-model run state; runs persisted (report + targets).
• CLI — coin-gym run|score|compare|improve|profiles.

The baseline-vs-team trade-off is demonstrable on the bundled sample: identical reach (60%) but the team's abstention gate lifts precision from 60% → 100% (baseline R:3/W:2, team R:3/A:2).

Merge-ready evidence

1. qa-team / gadugi scenarios — tests/gadugi/coin-gym-harness.{yaml,sh}: outside-in coverage of the full pipeline (run/score/compare/improve/profiles), the baseline-vs-team precision trade-off, the overfitting gate, offline-scaffold labelling, and usage-on-error. Hermetic, network-free.

• gadugi-test validate -f tests/gadugi/coin-gym-harness.yaml → ✓ valid.
• gadugi-test run -d tests/gadugi -s coin-gym-harness → ✓ Passed: 1 / Failed: 0 (command exit 0).

2. Docs — docs/howto/run-the-coin-gym-harness.md (new, wired into mkdocs nav under How-To Guides); research doc phase table + §3.5 CLI sketch reconciled to the implemented CLI. mkdocs build --strict → clean (no orphan/dead-link warnings).

3. Quality-audit — 3 SEEK→VALIDATE→FIX cycles, ended clean.

• Cycle 1 (independent review): no high-confidence issues.
• Cycle 2 (independent review): found hollow-offline-run, empty-target-set, missing offline-note in score/compare, run-id collisions, and profile-isolation gaps → all fixed (manifest validation, print_offline_note, RUN_SEQ + no-overwrite save_run, per-model ensure_profile rejection, --profile/run-id path-traversal hardening).
• Cycle 3 (independent review): found one residual (disjoint script keys → all-N hollow run) → fixed + test added. Final cycle: no high-confidence findings.

4. CI-equivalent, 100% green locally:

• cargo test --lib coin_gym → 73 passed, 0 failed.
• Full suite cargo test --all-features --locked → green in a clean env (the only local failures were pre-existing/environmental: the sandbox sets SIMARD_SCALING=auto, which overrides OodaConfig::max_concurrent_actions in unrelated ooda tests, and redirects CARGO_TARGET_DIR so the e2e test's hardcoded target/debug/simard path is absent — both pass in clean CI and are untouched by this diff).
• cargo clippy --all-targets --all-features --locked -- -D warnings → clean.
• cargo fmt --check → clean. Rust-only gate → passed. cargo build --bin simard --no-default-features compiles. Pre-commit + pre-push hooks passed on push.

6. Focused diff — additive: new src/coin_gym/ module + coin-gym bin + pub mod/2 re-exports in lib.rs + one [[bin]] in Cargo.toml + docs + gadugi scenario. No unrelated edits.

Scope / follow-ups (unchecked on #2713)

• Phase 3 — provision azlin VM + Docker, pull a COIN snapshot, wire the real coin evaluate executor (HIGH-RISK, operator-gated).
• Phase 5 — live self-improvement loop (apply tactic → verify on held-out fresh targets → keep-or-roll-back; durable tactic memory).

Closes part of #2713 (Phase 4 only).

[github-actions]

📊 Coverage Summary

Generated by cargo llvm-cov --workspace --summary-only (nightly, excluding test files)

Module	Lines	Covered	Coverage
Total	149874	125675	83.9%

_{Coverage data from CI run. Test files matching tests?/ are excluded from line counts.}

[rysweet]

CI status (merge-ready gate #4)

All code-validating CI jobs are green on the PR-event run (28829091204):
pre-commit (cargo test + clippy -D warnings + rust-only gate + --no-default-features build), build, coverage, install-real, e2e-dashboard, cargo-vet, npm-audit — all ✅.

The only red checks are cargo-audit / cargo-deny, failing on RUSTSEC-2026-0204 (crossbeam-epoch — "Invalid pointer dereference in fmt::Pointer"). This is:

• External to this diff. crossbeam-epoch is a transitive dependency (549-crate graph); this PR does not modify Cargo.lock (the lockfile delta shown vs. origin/main is only main's simard 0.27.0 → 0.28.0 version bump landed after this branch point).
• A freshly-published advisory that landed mid-CI: the earlier push-triggered run's cargo-audit failed on it while the slightly-earlier PR-event run passed (identical Cargo.lock); a fresh cargo audit now reproduces the failure locally. It will therefore fail on main and every open PR until remediated.
• Fleet-level, not PR-level. Remediation is cargo update -p crossbeam-epoch to a patched release (or an advisory ignore in deny.toml + cargo-audit config), which is out of scope for this COIN Gym Phase-4 change and owned by the CI-health steward (cf. feat(ci-health): codify a precise, reproducible governed-fleet CI-health sweep #2733).

Not self-merging while these supply-chain gates are red. Once the fleet-wide advisory is addressed, this PR is clean-mergeable (mergeable: MERGEABLE).