feat: auto-select embedding-capable agents - #789
Conversation
|
Warning Review limit reachedNext included review available in 47 minutes. View limit detailsLimit details: You’ve used the included review currently available. You've used all free OSS reviews for now. Wait for the free limit to reset to keep reviewing this public repository. Review configuration: ⚙️ Run configurationConfiguration used: Organization UI Review profile: CHILL Plan: Pro Plus Run ID: 📒 Files selected for processing (18)
📝 WalkthroughWalkthrough임베딩 API에서 모델 생략과 capability 기반 자동 선택을 지원한다. 모델 discovery는 capability를 보존한다. CLI는 chat 모델을 자동 검색한다. 배치 응답과 비용 집계는 해석된 모델을 유지한다. Changes임베딩 모델 선택과 자동 검색
Estimated code review effort: 4 (Complex) | ~45 minutes Merge Risk: 🟡 Moderate · up to Auto-selection changes embedding routing, but the current implementation can overwrite operator-managed discovered-agent settings on restart and return the wrong model identity for empty batches; explicit null or blank model values also bypass the intended auto-selection behavior. These are bounded but concrete default-behavior and API-correctness risks, so the PR is not merge-ready until they are fixed or explicitly accepted. Sequence Diagram(s)sequenceDiagram
participant Client
participant EmbeddingEndpoint
participant TaskOrchestrator
participant ModelAgent
Client->>EmbeddingEndpoint: 임베딩 요청 전송
EmbeddingEndpoint->>TaskOrchestrator: embedding capability 에이전트 선택
TaskOrchestrator->>ModelAgent: 활성 상태와 capability 검증
ModelAgent-->>EmbeddingEndpoint: 해석된 모델 반환
EmbeddingEndpoint-->>Client: 임베딩 응답 또는 503 embedding_unavailable
🚥 Pre-merge checks | ✅ 5✅ Passed checks (5 passed)
✨ Finishing Touches📝 Generate docstrings
🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
|
Current-head proof for
The hosted required checks are the remaining current-head evidence. Auto-merge will remain protected by the repository ruleset; no self-approval, admin merge, or force-push was used. |
|
Current-head refresh for
Hosted required checks remain the release evidence. Protected auto-merge is enabled; no self-approval, admin merge, or force-push was used. |
|
Review pass on exact current HEAD
@opencode-agent please review this exact HEAD. |
|
@opencode-agent review exact current HEAD 3a80d91. Review the omitted-model embedding capability contract and publish a formal review. |
|
Please review current exact HEAD |
|
@devin review exact current HEAD |
|
Exact-head local verification for
Hosted required Checks and an independent protected approval remain the merge gates; no bypass or self-approval used. |
|
@devin review exact current HEAD `3a80d91b8c879e57d30ab87af664546b8712fb15` against `main`; focus on capability-constrained embedding auto-selection, explicit-model validation, batch parity, and provider/cost attribution. |
|
@devin review exact current HEAD 02e13a8fdd53446a77af4ea8a25559f7f34a10af. The automatic embedding selection ADR now uses unique number 0024 (the previous 0012 collided with the accepted gateway-only ADR). The changed-contract suite passes (60 passed) and changed-file Ruff/diff checks pass; revalidate current implementation and documentation only. |
|
Correction: the exact current HEAD is 02e13a8 (the earlier short SHA expansion in the request was not authoritative). Review only this full SHA. The automatic embedding selection ADR is uniquely numbered 0024; changed-contract tests passed (60 passed) and changed-file Ruff/diff checks passed. |
|
Current exact HEAD
@devin-ai-integration please revalidate this exact HEAD. Normal auto-squash merge remains armed; no approval or bypass is used. |
|
Current exact-head review refresh: HEAD is 930bce1. Review findings and protected Checks must be evaluated against this SHA; no prior-head evidence is reused. |
|
Exact-head review pass for
The PR body’s older SHA is stale; use only this current HEAD. No new local defect or unresolved review thread was found. Hosted protected approval is still required. @opencode-agent review exact current HEAD |
|
Validation for the live current HEAD
@opencode-agent Review only exact current HEAD |
|
Exact current HEAD 930bce1 was revalidated for automatic embedding-capable model selection.
Please review and run protected Checks for this exact HEAD only. |
|
Exact current HEAD fb46918 was revalidated after the concurrent embedding-selection update.
Please review only this exact HEAD. |
Exact-head validation — PR #789
@opencode-agent please review only exact current HEAD |
|
Exact current HEAD fb46918 was reviewed for omitted embedding-model selection, explicit capability validation, sync/batch HTTP behavior, and startup discovery. Focused embedding/API/startup/security proof: 41 passed; Ruff, compileall, and diff-check passed. No reproducible defect or unresolved current review thread was found. Hosted Checks and protected independent approval remain required; no merge or release authority is claimed. |
|
Exact-head queue review for |
|
Exact-head review update for |
![devin-ai-integration[bot]](https://avatars.githubusercontent.com/in/811515?s=60&v=4){kind=small}
This comment was marked as resolved.
This comment was marked as resolved.
Sorry, something went wrong.
This comment was marked as resolved.
This comment was marked as resolved.
Sorry, something went wrong.
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
This comment was marked as resolved.
This comment was marked as resolved.
Sorry, something went wrong.
This comment was marked as resolved.
This comment was marked as resolved.
Sorry, something went wrong.
0eaf2a5 to
9f9e18b
Compare
| capabilities=("chat",), | ||
| ), | ||
| ProviderModelSource( | ||
| provider_name="nvidia_nim_sub", | ||
| credential_name="NVIDIA_NIM_API_KEY_SUB", | ||
| list_url="https://integrate.api.nvidia.com/v1/models", | ||
| chat_base_url="https://integrate.api.nvidia.com/v1", | ||
| capabilities=("chat",), |
There was a problem hiding this comment.
🟡 NVIDIA models activated as chat agents without chat-only filtering
The nvidia_nim and nvidia_nim_sub sources declare capabilities=("chat",) while leaving their model-listing URL unfiltered, unlike openrouter (?output_modalities=text) and bytez (?task=chat). The NVIDIA listing returns embedding and reranking deployments too, so --auto-discover-model-agents enables those non-chat models as enabled chat agents.
Prompt for agents
The nvidia_nim and nvidia_nim_sub ProviderModelSource entries in contextual_orchestrator/model_discovery.py declare capabilities=("chat",) but their list_url (https://integrate.api.nvidia.com/v1/models) is not restricted to chat/text models, unlike openrouter (which appends ?output_modalities=text) and bytez (task_filter="chat"). Per ADR 0024, a source may only declare chat when its listing endpoint is explicitly limited to chat/text LLMs. Because _auto_discover_runtime_agents activates every discovered model whose capabilities contain chat, embedding/reranking deployments from the NVIDIA catalog would be enabled as chat agents at startup. Either restrict the NVIDIA list_url to chat/text models (or add a NVIDIA-appropriate task/type filter) before declaring the chat capability, or drop the chat capability from these sources so they follow the same conservative default as the openai source.
Was this helpful? React with 👍 or 👎 to provide feedback.
| agents = [ | ||
| replace(agent_from_discovered(model), disabled=False) | ||
| for model in chat_models | ||
| if agent_id_for(model) not in existing_ids | ||
| ] |
There was a problem hiding this comment.
📝 Info: Auto-discovered chat agents score 0 for all workflow roles
Agents activated by _auto_discover_runtime_agents carry only ("discovered", "chat") tags. No role in ROLE_TAGS maps to chat, so _score_agent gives them role_score 0 for thinker/worker/verifier/synthesizer; they rank purely by priority and never outrank a tagged seed agent in conducted workflows.
Was this helpful? React with 👍 or 👎 to provide feedback.
|
Merge-gate evidence (2026-08-24): Review fixes applied and pushed (see commits); all required checks green on current head except |
1 participant
Summary
Allow sync and batch embedding requests to omit model, then resolve an enabled embedding-capable agent through the existing orchestrator ranking policy while preserving explicit-model validation and fail-closed embedding_unavailable behavior.
Discovery preserves only explicitly declared source capabilities into runtime agent tags. It does not infer embedding support from provider order or model names. Pending and completed batch documents retain the resolved top-level model identity.
Consumer contract
Current-head verification
This PR does not change chat/completions model validation or provider credentials. Do not merge without independent approval, resolved threads, and terminal required Checks.
Summary by CodeRabbit
새 기능
503 embedding_unavailable을 반환합니다.문서
테스트