feat(v0.4-beta): learned-selector seam (model-free, shadow-only)

[snchimata]

Summary

Starts v0.4-beta (Phase 8): the committed, model-free seam that lets a locally-trained keep/drop selector plug into the baseline harness as just another SELECTORS entry — with no model code or weights in git. Originally stacked on PR #4 (now merged into develop, carrying the paired-task corpus); still shadow-only, nothing served. (Re-opened as a fresh PR targeting develop directly — the original PR #5 was auto-closed when its base branch, feat/v0.4-alpha-corpus-baselines, was deleted after PR #4 merged; same branch, same commits, no content lost.)

What's in it (3 files, model-free)

• run_baselines.py — _load_learned_selectors(): discovers an optional local module via $TOKENFOLD_LEARNED_MODULE (default learned.selector), reads its LEARNED_SELECTORS dict of (units, query) -> list[float] score functions, and registers them into SELECTORS. When the ML stack / module / weights are absent the import fails and the selector is skipped cleanly — the exact optional-dep pattern already used for tiktoken and the tokenfold CLI. Report + summary now label learned vs. deterministic selectors.
• test_learned_selector_wiring.py: stdlib, assert-based, runs with no ML installed. Proves skip-when-absent never raises, the real register path flows through build_report/run_gate, a baseline-name collision is rejected, and any learned score function still passes the hard gates.
• .gitignore: /eval/learned/ — model/training code + weights stay local, never committed.

Contract (why this is safe)

Per model-research.md: the learned selector emits scores only. The deterministic critical-atom forcing + ceiling allocator + byte-copy assembly own the output, so 100% critical-atom survival and the token ceiling stay structural — independent of what the model scores. --gate proves this holds for the learned selector too.

Integrity guard (from review)

A 4-lens adversarial review flagged that a learned selector named like a deterministic baseline (e.g. bm25) would silently overwrite that baseline, run under its name, yet be reported as deterministic. The seam now fails loud on any name collision with a deterministic baseline, and a test asserts it. A second finding — the register line was never exercised end-to-end — is fixed with a test that drives the actual SELECTORS.update(...) through build_report/run_gate.

The bar to beat

bm25 = 0.82 task success @ 0.25 token ratio (v0.4-alpha finding). The local (gitignored) frozen-Granite probe trains + runs against this bar once the ML stack is installed; "stop if it can't beat bm25."

Testing

python eval/test_learned_selector_wiring.py   # checks pass (model-free)
python eval/run_baselines.py --gate           # gate: pass
python eval/run_baselines.py                  # "learned selectors: none (ML absent, skipped)"

git check-ignore confirms eval/learned/{selector,train_probe}.py + weights/head.pt are excluded; the index holds only the 3 files above.