Evaluate fact and episode models against the reviewed corpus

[dcellison]

Outcome

Select fact-extraction, episode-classification, and episode-generation models using the reviewed corpus rather than assuming a stronger or more expensive model is better.

Requirements

• Replay identical inputs and candidate context through current cheap-tier models and approved stronger candidates.
• Measure strict factual precision, useful recall, over-extraction, fragmentation, consolidation intent, scope, episode classification, structured-output reliability, latency, and provider failures.
• Evaluate fact extraction and episode generation independently.
• Produce a versioned comparison report and explicit promotion threshold.
• Permit per-role defaults or policy-bounded overrides without changing any conversational agent's selected/default model.
• Fail closed when the selected role model is unavailable or outside runtime authority.

Acceptance

• The production choice is justified by recorded metrics.
• Re-running the evaluation against the same corpus is deterministic apart from documented provider variance.
• A model promotion has a rollback path and preserves historical model provenance.

[dcellison]

Reopened during epic reconciliation because the model-evaluation machinery is installed, but this issue's outcome and acceptance criteria have not yet been met.

Installed diagnostics currently report comparisons=0. No versioned reviewed comparison has yet justified the production fact-extraction, episode-classification, or episode-generation model choices against the same immutable corpus. This issue remains open until the approved model arms are evaluated, reviewed blind, scored against explicit promotion thresholds, and a production/rollback decision is recorded.

[dcellison]

This issue is currently blocked by #1706's production-data readiness gate.

The model-evaluation workflow requires an immutable snapshot plus a sealed human review. #1706 currently has only 10 fact-extraction receipts, no episode-generation receipts, no stored/replaced outputs, and no qualified reviewed baseline. Installed diagnostics therefore correctly report comparisons=0.

Do not run or accept a production model comparison against an unqualified interim corpus. Once #1706 produces a qualified sealed review, this issue can replay identical reviewed cases through the authorized model arms, perform the blind review, score the results, and record the explicit production and rollback decision.