Sidecar preflight: Layer 1's 160s worst case excludes discover_all_models()'s own sequential network time

[seonghobae]

Background

Devin Review, on PR #1452 (ADR-0005's implementation), verified against current code and the vendored
contextual-orchestrator source: the 180s healthz-readiness watchdog
(scripts/ci/contextual_orchestrator_review_sidecar.sh, the until curl .../healthz ...; do ... i -ge 180 ... loop) starts polling immediately after the launcher process (contextual_orchestrator_review_launcher.py main()) is spawned — and that process's first substantial work is
contextual_orchestrator.model_discovery.discover_all_models(), not preflight probing. Layer 1's own
"160s worst case, under the 180s ceiling" arithmetic (REVIEW_PREFLIGHT_MAX_ESCALATIONS's docstring
comment, scripts/ci/contextual_orchestrator_review_launcher.py around line 63) accounts only for the
probing/escalation phase (REVIEW_PREFLIGHT_MAX_TOTAL_ROUTES * REVIEW_PREFLIGHT_TIMEOUT_SECONDS + REVIEW_PREFLIGHT_MAX_ESCALATIONS * REVIEW_PREFLIGHT_TIMEOUT_SECONDS = 160s) — it does not include
discovery's own time, even though both phases run inside the same 180s watchdog.

Verified against the actual vendored source (not assumed)

contextual_orchestrator/model_discovery.py (checked directly in the vendored checkout):

• DISCOVERY_TIMEOUT_SECONDS = 15.0, applied per HTTP call via urllib.request.urlopen(..., timeout=timeout).
• discover_all_models() makes these calls sequentially, not concurrently:
  1. One _fetch_json(_MODELS_DEV_URL, ...) call, if any source with models_dev_provider_id has a
     registered credential (true for this sidecar's pool: openai, nvidia_nim, nvidia_nim_sub all
     declare it and all have credentials the sidecar registers).
  2. One discover_provider_models(source, ...) call per entry in PROVIDER_MODEL_SOURCES (6
     entries: openai, openrouter, opencode_zen, nvidia_nim, nvidia_nim_sub, bytez) — a source
     with no registered credential returns [] immediately with no network call
     (if not api_key: return []), but this sidecar registers 5 of the 6 (all but opencode_zen), so 5
     of these are real, sequential network calls.
  3. One final _openrouter_zdr_model_ids(...) call (openrouter has a credential here, so this runs too).
• Worst case: 7 sequential calls × up to 15s each = up to ~105s, entirely before Layer 1 probing
  starts, inside the same 180s watchdog that also has to fit the 160s probing/escalation worst case.
  Combined real worst case is therefore up to ~265s, not the ~160s the current comments imply — a
  slow-but-not-fully-hung discovery pass alone could exhaust the watchdog before probing gets any of its
  own 160s budget.

Options considered (not evaluated to a decision)

1. Thread one monotonic remaining-time deadline through discovery (discover_all_models(timeout=...)
   already accepts a timeout kwarg on the launcher's side — no vendored-source change needed) and
   probing, so the two phases share one real budget instead of each assuming the full window.
2. Scale down probe/escalation route count or the escalation budget based on however much of the 180s
   discovery already consumed.
3. Extend the 180s watchdog to a new, evidence-justified figure that honestly covers the full sequence
   (discovery + probing) — needs real observed timing, not an inspection-only guess (this org's ADR-0005
   explicitly rejects "heuristics and rules of thumb" as a justification standard).

Ask

Design and land a fix that makes the actual, combined worst-case bound explicit and correct (or justify
extending the watchdog with real telemetry). Not blocking PR #1452's fix of the 7 verified Devin Review
findings already in that PR — flagged there as architecturally significant enough to need its own pass
rather than a guessed patch, per this org's convergence convention (initial values from precedent,
refinement from telemetry, never from inspection alone).

Cross-ref: ADR-0005 (docs/adr/0005-sidecar-preflight-token-budget.md), PR #1452, PR #1449,
ContextualWisdomLab/contextual-orchestrator's model_discovery.py.

[seonghobae]

Decision (2026-08-30): accepted as a known, tracked residual risk — not blocking PR #1452

Reviewed with the coordinator. The verified numbers stand: discovery's own sequential worst case is up to
~105s (7 sequential HTTP calls at up to DISCOVERY_TIMEOUT_SECONDS = 15s each, per the vendored
contextual_orchestrator.model_discovery source), and Layer 1 probing's own worst case is 160s, both
inside the same 180s healthz-readiness watchdog — combined worst case up to ~265s, not the ~160s the
prior comments implied.

Not blocking PR #1452 on this, for the same reasoning as ContextualWisdomLab/.github#1454:

• This PR's design is a genuine, verified improvement over the status quo (no diagnostic retry at all,
  reproducing the 120s-timeout bug repeatedly) — it doesn't need to close every tail-case failure mode to
  be worth merging.
• The failure mode this issue describes requires TWO unlikely things to coincide in one run: discovery
  running unusually slowly (near its own worst case) AND probing separately needing close to its full
  escalation budget. That's a tail case, not the common path.
• Per this org's own convergence/no-heuristics convention (ADR-0005 explicitly rejects picking a number
  from inspection alone), there's no real timing telemetry yet for discovery's actual observed duration
  the way there is for probing — picking a specific fix (shared deadline, scaled-down probing, or a
  justified watchdog extension) without that evidence would itself be exactly the kind of unjustified
  guess this org's convention exists to prevent.
• The inaccurate "160s, safely under 180s" framing is already corrected in place (source comments in
  contextual_orchestrator_review_launcher.py's REVIEW_PREFLIGHT_MAX_ESCALATIONS and the sidecar
  script's 180s watchdog loop both now state the real combined worst case and point here), so this is no
  longer a silent, wrong safety-margin claim — it's an honest, tracked, open gap.

Stays open as an accepted, tracked, non-blocking residual-risk item. Follow-up (real timing telemetry
from hosted runs, then a specific fix) happens here when someone picks it up.

Cross-referenced from docs/adr/0005-sidecar-preflight-token-budget.md,
scripts/ci/contextual_orchestrator_review_launcher.py, and
scripts/ci/contextual_orchestrator_review_sidecar.sh; the corresponding PR #1452 review thread is
resolved with a pointer to this comment.

---

Generated by Claude Code