Epic: Agent runner router — programs → routers → (runner, model) pairs

[gewenyu99]

Epic: Agent runner router — programs → routers → (runner, model) pairs

Summary

De-risk the wizard's dependency on a single agent loop (@anthropic-ai/claude-agent-sdk) by adding
capability through a central config map + named insertion points, not by reworking existing flags. A
config map gives every program its defaults (router, runner, model); control is then asserted at
clearly-defined insertion points (middleware), where each flag is its own middleware. The headline
deliverable is a second runner, pi (pi.dev, @earendil-works/pi-coding-agent),
runnable against the same gateway / skills / TUI for A/B against anthropic.

Purely additive. Existing flags are not touched — each becomes a middleware named for it. With no flag
set, every program resolves to linear / anthropic / sonnet, so today's behavior is identical.

Architecture — config map, two registries, insertion points

Concept	Values	Role
Router	linear, orchestrator	control-flow shape — how many leaves run and which pair each gets
Runner	anthropic, pi	leaf engine — owns an agent loop, takes a resolved model id
Pair	{ runner, model }	what a leaf resolves to / the orchestrator requests per sub-task

• RUNNERS = { anthropic, pi }, ROUTERS = { linear, orchestrator }, MODELS = alias→gateway-id map
  (sonnet→claude-sonnet-4-6, opus→claude-opus-4-8) — replaces the 4 hardcoded model constants.
• Config map — ROUTES (the base, just a map): program → { router, runner, model } with optional
  per-role pair overrides for the orchestrator; absent → DEFAULT_ROUTE = { linear, anthropic, sonnet }.
  The new capability is per-program model selection (doesn't exist today). No default-resolver
  functions — reading the map is the base decision.
• Insertion points (middleware): each decision flows through an ordered chain terminating in the map —
  resolveRouter and resolvePair. A middleware defers (next()) or asserts.

Control flow:

runProgram(program)
 ├─ router = ROUTERS[ resolveRouter(ctx) ]              // INSERTION POINT 1
 └─ router.run → runLeaf(ctx, role)
        └─ pair = resolvePair(ctx, role)                // INSERTION POINT 2, per leaf
             └─ RUNNERS[anthropic|pi].run({ model: MODELS[pair.model] })

Flags — existing, each its own middleware

The resolver introduces no flags and renames none. Each flag is a middleware named for it:

Flag	Type	Effect (unchanged)	Middleware
wizard-orchestrator	bool	picks the orchestrator path	wizardOrchestrator in ROUTER_MIDDLEWARE
wizard-runner	anthropic|pi	picks the runner	wizardRunner in PAIR_MIDDLEWARE (overrides the pair's runner; model stays from config)

• One flag ⇒ one named middleware. A future wizard-foo → wizardFoo added to the relevant chain;
  nothing else changes. Future controls (cost caps, per-org policy, A/B) are middleware the same way, and
  automatically cover orchestrator leaves (same resolvePair).
• Revert = wizard-runner=anthropic (today's default).

Design notes (Danilo's review): declarative, compositional slots (pull a middleware / map row / registry
entry in or out — never a switch) and belt-and-suspenders defaults (no flag → control). Design doc:
workbench/pi-runner-router-design.md.

Why Pi

• It's a coding agent — owns the tool-calling loop and ships its own Read/Write/Edit/Bash.
• Native file-based skills — SKILL.md + progressive disclosure from scanned dirs (context-mill model).
• Speaks our gateway directly — registered anthropic-messages provider (bearer, Bedrock-fallback +
  metadata/flag headers).
• In-process SDK — createAgentSession → session.prompt → session.subscribe; tools via defineTool.

Sub-issues

• 01 — Selection surface: config map + insertion points (each flag its own middleware) #521 — 01 · Selection surface: config map + insertion points — (feat(runner): central runner plan + anthropic runner seam (#521) #692 plumbing + feat(runner): pi.dev runner — gateway provider, model from pair, in RUNNERS (#524) #693's wizardRunner middleware)
• 02 — Shared runner contract + adapters (gateway, MCP, prompt, tools, stream→TUI) #522 — 02 · Shared runner substrate — gateway, MCP, prompt, tools, stream→TUI; model from the resolved
  pair. (landed across feat(runner): central runner plan + anthropic runner seam (#521) #692–feat(runner): wizard tools as pi custom tools — real integration on pi (#524) #694)
• 03 — Vercel AI SDK runner (vercel) #523 — 03 · Vercel runner — dropped; closed in favor of pi.
• 04 — pi.dev runner (pi) #524 — 04 · pi runner — registers into RUNNERS, consumes the pair's model. (PRs feat(runner): pi.dev runner — gateway provider, model from pair, in RUNNERS (#524) #693, feat(runner): wizard tools as pi custom tools — real integration on pi (#524) #694)
• 05 — Security parity (canUseTool + YARA, fail-closed) for pi #525 — 05 · Security parity (canUseTool + YARA, fail-closed) for the pi runner — (PR feat(runner): fail-closed security parity on the pi backend (#525) #697)
• 06 — Subagent / Task dispatch parity for pi #526 — 06 · Subagent / Task dispatch parity for pi — (PR feat(runner): Task/todo + controlled subagents + logging parity on pi (#526) #698)
• (dropped) auth-isolation — neutralizeInheritedAgentSession (no-op in prod). fix(anthropic): don't inherit an outer Claude Code session for gateway auth #700 auto-merged by a
  branch-order quirk; fix(anthropic): don't inherit an outer Claude Code session for gateway auth #748 (the duplicate) closed. Re-add on a fresh branch if wanted.
• 07 — Rollout + per-runner observability (canary via wizard-runner) #527 — 07 · Rollout + per-runner observability (canary via wizard-runner) — (pending)
• 08 — Cross-runner parity test suite #528 — 08 · Cross-runner parity test suite — (pending)
• 09 — Decide, ramp, and decouple #529 — 09 · Decide, ramp, and decouple — (pending)
• (landed) pi perf parity — native ls/find/grep steering, skill-menu cache, batch tool calls, 1M context.
  (PR perf(pi): steer to native tools + cache skill menu #699)
• (landed) pi PostHog data-ops via real MCP — dashboard/insight tools + scrubbed-env lockdown. (PR feat(pi): real PostHog MCP dashboard, env lockdown, perf parity #701)

Stack mapping (the actual tiny-PR stack, on latest main)

Reshaped in place onto the existing branches — same PR numbers, each tiny + single-concern, building green
at every step. Land order is the stack order:

PR	Branch	Content
#692	pi/01-seam-flag	Central runner plan + anthropic seam. runner-plan.ts (RUNNERS/ROUTERS/MODELS/ROUTES/DEFAULT_ROUTE, resolvePair over a middleware chain — empty here), backends/{types,anthropic}.ts, linear.ts dispatches through the runner; the 4 model literals → MODELS/DEFAULT_AGENT_MODEL; re-add WIZARD_RUNNER_FLAG_KEY. Behavior identical (every program → linear/anthropic/sonnet).
#693	pi/02-pi-backend	pi.dev runner. piBackend registered in RUNNERS; wizardRunner pair-middleware (reads the existing wizard-runner flag); pi.ts consumes the pair's model + infers the gateway transport (claude → anthropic-messages, openai/* → openai-completions); gpt5 model alias; pi deps.
#694	pi/03-wizard-tools-on-pi	Wizard env-file tools as pi custom tools.
#697	pi/sec-fence	Fail-closed security parity (canUseTool + YARA) via a pi tool-execution extension; brings yara-scanner.
#698	pi/tasks-subagents	Task/todo tools (live TUI) + controlled read-only subagents.
#699	pi/perf-tuning	Anti-spiral runtime notes (read-not-shell, sync installs), init banner.
#701	pi/mcp-dashboard	Real PostHog MCP dashboard via pi's MCP extension + scrubbed-env lockdown; 1M context. Based directly on pi/perf-tuning.

Auth-isolation dropped from the stack. neutralizeInheritedAgentSession (anthropic-side, no-op in
prod — only matters when the wizard runs inside another Claude session, i.e. the test harness, which already
strips CLAUDE* env itself) was its own PR but became pure noise: GitHub auto-merged it as #700 when a
branch-order swap made its head an ancestor of its base, leaving a permanently-merged entry mid-stack plus a
duplicate open PR (#748) on the same branch. Both closed/orphaned; the work can be re-added cleanly on a
fresh branch if wanted. The stack is now 7 tiny PRs, a clean linear chain, CI green at every step.

Implementation note vs the design above: the pair insertion point + wizardRunner middleware are in
(#692 plumbing, #693 middleware); the router side keeps the existing isOrchestratorEnabled fork as-is
(the resolveRouter/wizardOrchestrator middleware + orchestratorRouter registration remain the future
additive step, not in this stack). Live snapshot evidence (pi/anthropic/gpt-5/orchestrator) is on #693.

CI note: the pi deps (@earendil-works/*) require Node ≥22.19, so #693 drops the obsolete 20.20.0 matrix
entry + bumps engines to >=22.22.0 (the wizard already targets Node 24).

Open questions

• Does Pi resolve a custom-provider model without interactive auth in headless/CI? (handled in 04 — pi.dev runner (pi) #524/feat(pi): real PostHog MCP dashboard, env lockdown, perf parity #701.)
• Precedence/order of PAIR_MIDDLEWARE as more flags are added.

Local testing (mprocs)

• Build channel — WIZARD_BUILD_NODE_ENV=ci keeps WIZARD_CI_FLAG_OVERRIDES live AND targets prod
  cloud. A wizard-build-ci mproc builds it.
• Selection — the existing flags, via WIZARD_CI_FLAG_OVERRIDES='{"wizard-runner":"pi"}' (and/or
  "wizard-orchestrator":"true") in the workbench .env.
• Observability — diagnostics to the log file; each run logs its resolved router + runner + model.

Acceptance

• pi runs a real integration end-to-end when wizard-runner=pi, default-off; anthropic unchanged.
• Existing flags behave exactly as before (now as middleware); defaults reproduce today's behavior.
• Skills load via Pi's native discovery; security parity (canUseTool + YARA) before any non-zero ramp.
• A documented, repeatable mprocs test flow.

Rebase reality (planning workflow, 2026-06-26)

Main moved 29 commits past the stack base (#688) and diverged in the reshape's exact files, so the stack is
re-authored on top of latest main, not rebased (rebasing would silently drop main's inline additions).
Per-file reconciliation the reshape must honor:

• linear.ts (biggest trap): start from main's version verbatim. Main added, inline on the anthropic
  path, richLinks→createWizardAskBridge, orgAiDataProcessingApproved+teamProductOptIns→assemblePrompt,
  and emitStepEvents: config.trackStepProgress ?? false→executeAgent. The only delta from main is
  replacing the inline initializeAgent/executeAgent block with the resolved-runner dispatch; those three
  fields must be threaded through BackendRunInputs into anthropicBackend or the path regresses silently.
• constants.ts: WIZARD_VARIANT_FLAG_KEY / WIZARD_VARIANTS are tombstoned on main — do NOT
  re-add. DEFAULT_AGENT_MODEL (= 'claude-sonnet-4-6') + HAIKU_MODEL now exist on main; the model-literal
  consolidation uses DEFAULT_AGENT_MODEL. Re-add only WIZARD_RUNNER_FLAG_KEY (first commit, before any
  importer).
• bootstrap.ts: main replaced buildWizardMetadata→buildRunTags; analytics.setTag('variant',…) moved
  to stampVariant in runner/index.ts — do NOT re-add it to bootstrap.
• runner/index.ts: main added stampVariant + WizardVariant enum (BASE/ORCHESTRATOR/PI). Keep
  stampVariant; wire resolveRouter here. WizardVariant.PI stamping lands in feat(runner): pi.dev runner — gateway provider, model from pair, in RUNNERS (#524) #693 once the resolved
  runner is pi.
• agent-interface.ts: main already changed the model literal → config.modelOverride ?? DEFAULT_AGENT_MODEL
  and added the warlock/YARA/triage overhaul (+204 lines). The auth-isolation (fix(anthropic): don't inherit an outer Claude Code session for gateway auth #700) neutralizeInheritedAgentSession
  must be re-authored as a new commit (line-shifted ~40 lines; a cherry-pick rejects).
• Model literals: only mcp-prompt-streaming.ts + agent-prompt-loader.ts still hold the bare literal →
  point both at DEFAULT_AGENT_MODEL (agent-interface already does on main).
• pi.ts MODEL_ID: stays through #692a/#692b; only feat(runner): pi.dev runner — gateway provider, model from pair, in RUNNERS (#524) #693 removes it (after model is threaded via the
  pair) — removing earlier leaves an intermediate state with no model.
• Second-model capability: the gateway serves openai/gpt-5 on a separate /v1/chat/completions
  endpoint; pi.ts currently hardcodes api:'anthropic-messages', so feat(runner): pi.dev runner — gateway provider, model from pair, in RUNNERS (#524) #693 must infer the api shape from the
  model id (openai/ → openai-completions, baseUrl keeps /v1; anthropic → strip /v1).

D5 (two PRs vs two commits) is resolved: #692a and #692b are separate PRs. Land order unchanged.