fix: recover passthrough provider failover

[seonghobae]

Summary

• fail over virtual-model tools, structured-output, and Responses passthrough after transient or stale-model failures
• keep explicitly requested concrete models sticky and fail closed on caller/configuration errors
• preserve provider-shaped payloads, local Responses conversion, per-provider reasoning-effort mapping, and one-attempt-per-provider replay safety

Evidence

• uv run --frozen --extra test --with pytest pytest -q -> 1886 passed
• focused passthrough/Responses/capability suite -> 39 passed
• git diff --check -> clean

Grounding

• RFC 9110 section 9.2.2 replay safety
• existing RouteLLM, Hybrid LLM, and FrugalGPT research mapping in docs/doctoring/provider-diverse-discovery-routing.md

Advances #846 (passthrough failover slice only).

---

Summary by CodeRabbit

• 새 기능

  • 자동·무료 모델과 모델 그룹을 지원하며, 적합한 공급자로 자동 전환합니다.
  • 구조화된 출력, Responses API, 도구 호출, 멀티모달 입력을 지원합니다.
  • 요청 유형에 따라 conducted·passthrough 분석 이벤트를 구분합니다.

• 개선 사항

  • 공급자 사용량이 없어도 예상 토큰과 비용 원장을 유지합니다.
  • 비용 기록에 측정 상태를 표시하고, 여러 통화는 환산 없이 통화별로 제공합니다.
  • 지원되지 않는 기능이나 이미지 요청에 더 명확한 오류를 반환합니다.
  • 무료 모델 요청은 무료 공급자만 사용하도록 개선했습니다.

[coderabbitai]

Note

Reviews paused

It looks like this branch is under active development. To avoid overwhelming you with review comments due to an influx of new commits, CodeRabbit has automatically paused this review. You can configure this behavior by changing the reviews.auto_review.auto_pause_after_reviewed_commits setting.

Use the following commands to manage reviews:

• @coderabbitai resume to resume automatic reviews.
• @coderabbitai review to trigger a single review.

Use the checkboxes below for quick actions:

• ▶️ Resume reviews
• 🔍 Trigger review

📝 Walkthrough

Walkthrough

모델 그룹과 capability 기반 라우팅을 확장했습니다. 가상 모델의 provider failover를 제한된 오류에 적용했습니다. Structured Chat과 Responses 요청을 orchestration workflow로 처리하고, provider usage와 추정 usage를 cost ledger에 기록합니다.

Changes

Provider orchestration 및 failover

Layer / File(s)	Summary
모델 그룹과 capability 라우팅 contextual_orchestrator/orchestrator.py, tests/test_openai_passthrough.py, tests/test_local_mlx.py, tests/test_chat_response_format_http_honesty.py	자동·무료 모델, response_format, vision capability 조건을 모델 선택과 workflow 후보에 적용했습니다. Responses 입력과 이미지 content 변환도 처리합니다.
Passthrough 오류 분류와 provider 순회 contextual_orchestrator/orchestrator.py, tests/test_passthrough_provider_failover.py, docs/doctoring/provider-diverse-discovery-routing.md	Omit-equivalent 제어값은 plain passthrough로 처리합니다. 허용된 transient HTTP·stale 모델·일시적 DNS 오류에서 provider를 한 번씩 시도합니다. 명시적 모델과 모호한 timeout은 fail-closed로 처리합니다.

Structured 및 Responses orchestration

Layer / File(s)	Summary
Evidence workflow와 synthesis contextual_orchestrator/orchestrator.py, contextual_orchestrator/server.py, docs/planning/adrs/0035-structured-provider-orchestration.md, tests/test_openai_passthrough.py, tests/test_responses_temperature_top_p_http_honesty.py, tests/test_analytics_runtime.py	Structured Chat과 비도구 Responses 요청을 evidence 호출과 단일 provider-native synthesis 호출로 처리합니다. 요청 형식, sampling 설정, 멀티모달 입력, lineage 및 분석 이벤트를 유지합니다.

Usage provenance

Layer / File(s)	Summary
측정 상태와 비용 기록 contextual_orchestrator/cost_ledger.py, contextual_orchestrator/cost_router.py, tests/test_cost_ledger.py, tests/test_cost_router.py, tests/test_cost_review_server.py, tests/test_distributed_cache_truth_and_isolation.py	measured, estimated, unavailable 상태를 ledger와 SQL 저장·조회 흐름에 추가했습니다. Provider usage가 없으면 prompt 중복을 피하는 token-counting fallback을 사용합니다. 다중 통화 비용은 MIXED 상태와 통화별 구성요소로 반환합니다.

검증 및 테스트 실행

Layer / File(s)	Summary
분석·계약 테스트와 테스트 환경 contextual_orchestrator/server.py, tests/test_analytics_runtime.py, tests/test_telemetry.py, Makefile, tests/test_repository_security_metadata.py, CHANGELOG.md	Chat과 Responses 분석 이벤트 구분, telemetry mock 인자, 로컬 테스트 requirements 및 변경 기록을 갱신했습니다.

Estimated code review effort: 4 (Complex) | ~60 minutes

Merge Risk: 🟠 High · up to 99166

The PR changes provider failover, replay behavior, structured-output routing, and usage accounting, but unresolved paths can skip healthy failover candidates, issue duplicate provider attempts, route explicit free-model requests to paid providers, or omit cost records. These correctness, availability, and billing risks should be fixed or explicitly accepted before merge.

Sequence Diagram(s)

sequenceDiagram
  participant Client
  participant Server
  participant Orchestrator
  participant Provider
  participant CostLedger
  Client->>Server: structured Chat 또는 Responses 요청
  Server->>Orchestrator: orchestration 요청 전달
  Orchestrator->>Provider: evidence 호출
  Provider-->>Orchestrator: evidence 응답과 usage
  Orchestrator->>Provider: synthesis 호출
  Provider-->>Orchestrator: 최종 응답과 usage
  Orchestrator->>CostLedger: usage와 measurement_status 기록
  CostLedger-->>Server: 비용 및 usage record 반환
  Server-->>Client: Chat 또는 Responses 응답

Loading

🚥 Pre-merge checks | ✅ 4 | ❌ 1

❌ Failed checks (1 warning)

Check name	Status	Explanation	Resolution
Docstring Coverage	⚠️ Warning	Docstring coverage is 67.06% which is insufficient. The required threshold is 80.00%. Docstring coverage is scoped to functions touched by this diff. Analyzed 85 functions across 17 files. (2 skipped:…	Write docstrings for the functions missing them to satisfy the coverage threshold.

✅ Passed checks (4 passed)

Check name	Status	Explanation
Description Check	✅ Passed	Check skipped - CodeRabbit’s high-level summary is enabled.
Title check	✅ Passed	제목은 가상 모델 passthrough 요청의 provider failover 복구라는 주요 변경을 정확하고 간결하게 설명합니다.
Linked Issues check	✅ Passed	Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check	✅ Passed	Check skipped because no linked issues were found for this pull request.

Full details: Docstring Coverage

Explanation

Docstring coverage is 67.06% which is insufficient. The required threshold is 80.00%. Docstring coverage is scoped to functions touched by this diff. Analyzed 85 functions across 17 files. (2 skipped: 1 unsupported, 1 too large.)

✨ Finishing Touches 💡 1

📝 Generate docstrings 💡

• Create stacked PR
• Commit on current branch

🧪 Generate unit tests (beta)

• Create PR with unit tests
• Commit unit tests in branch fix/recover-passthrough-provider-failover

---

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

• X
• Mastodon
• Reddit
• LinkedIn

_{Comment @coderabbitai help to get the list of available commands.}

[seonghobae]

Stack update: #853 merged normally into this branch at merge commit 7841f15ba42a97c5e8b4b5134f716b87c1c24d71. The exact head 71022d6c now includes conducted structured Chat/non-tool Responses synthesis, per-provider usage lineage, multimodal preservation, and the unique ADR 0034; the pre-stack full suite passed 1894 tests and the post-stack focused suite passed 51 tests. Hosted checks and independent exact-head review are being re-established.

[devin-ai-integration]

Devin Review found 0 new potential issues.