fix: use provider-backed embeddings for remote agents

[seonghobae]

Summary

• route configured remote embedding agents to their OpenAI-compatible provider endpoint
• preserve provider-reported embedding token usage
• retain deterministic local vectors only for mock/standalone pools
• accept the existing embedding selection ADR and clarify the provider boundary

Verification

• 29 passed across credential, batch embedding, routing-boundary, and cost-router tests
• git diff --check

This repairs the runtime boundary needed by LineageWeave Global Ask. It does not bypass protected review or checks.

---

Summary by CodeRabbit

• 새 기능

  • 임베딩 배치 작업이 비동기 처리, 상태 조회, 대기, 취소 및 재시작 복구를 지원합니다.
  • 입력별 비용 귀속과 메타데이터를 보존하며, 배치 제한·폴링 간격·보존 기간을 조회할 수 있습니다.
  • 임베딩 모델 자동 검색 및 원격 제공자 라우팅을 지원합니다.
  • Compose 병렬 실행을 위한 프로젝트명 설정을 제공합니다.

• 개선

  • 요청 기한이 재시도와 대체 제공자 처리 전반에 적용되며, 초과 시 명확한 오류를 반환합니다.
  • 제공자 장애 발생 시 임베딩 요청의 대체 처리가 강화되었습니다.

[coderabbitai]

Warning

Review limit reached

Next included review available in 52 minutes.

View limit details

Limit details: You’ve used the included review currently available.

You've used all free OSS reviews for now. Wait for the free limit to reset to keep reviewing this public repository.

Learn how review limits work.

Review configuration:

⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Pro Plus

Run ID: 501a3857-c14f-4916-a83a-069fd294e3e1

📥 Commits

Reviewing files that changed from the base of the PR and between 2a1810f and 77fd436.

📒 Files selected for processing (35)

• .github/workflows/tests.yml
• CHANGELOG.d/token-packer-utf8-boundary.md
• CHANGELOG.md
• Dockerfile
• Makefile
• README.md
• contextual_orchestrator/__init__.py
• contextual_orchestrator/__main__.py
• contextual_orchestrator/api_contract.py
• contextual_orchestrator/batch_job_registry.py
• contextual_orchestrator/batch_routing.py
• contextual_orchestrator/cost_ledger.py
• contextual_orchestrator/cost_router.py
• contextual_orchestrator/orchestrator.py
• contextual_orchestrator/reasoning_effort_profile.py
• contextual_orchestrator/server.py
• contextual_orchestrator/token_counting.py
• docs/adr/0003-cost-aware-sync-batch-routing.md
• docs/adr/0005-request-scoped-provider-deadline.md
• docs/planning/adrs/0002-explicit-local-mlx-evaluation.md
• rust/token_counter/src/lib.rs
• scripts/run_hash_locked_tests.sh
• tests/test_api_contract.py
• tests/test_batch_embeddings.py
• tests/test_batch_job_registry.py
• tests/test_batch_routing_boundaries_extra.py
• tests/test_cost_router_boundaries.py
• tests/test_healthz.py
• tests/test_model_judge.py
• tests/test_openai_passthrough.py
• tests/test_provider_reliability.py
• tests/test_repository_security_metadata.py
• tests/test_spend_analytics.py
• tests/test_token_counting_boundaries.py
• tests/test_token_counting_strategies.py

📝 Walkthrough

Walkthrough

임베딩 배치를 provider 기반 비동기 작업으로 변경했습니다. Rust 토큰 패커, 입력별 provenance, 요청 deadline, provider retry·failover 제어, streaming 오류 처리, 비용 meter 및 가속기 런타임 문서를 추가했습니다.

Changes

임베딩 라우팅

Layer / File(s)	Summary
비동기 provider 배치와 durable 상태 contextual_orchestrator/batch_routing.py, contextual_orchestrator/batch_job_registry.py	배치 제출, polling, 취소, 재시작 복구, claim lock 및 동시성 제어를 추가했습니다.
Capability 기반 라우팅과 provenance contextual_orchestrator/cost_router.py, contextual_orchestrator/cost_ledger.py, contextual_orchestrator/server.py	Provider 제한에 따라 입력을 분할하고, provider 사용량과 입력별 attribution·metadata를 저장합니다.
HTTP 계약과 검증 contextual_orchestrator/api_contract.py, tests/test_batch_embeddings.py, tests/test_cost_router.py	Capability endpoint, polling metadata, deduplication, failover, cancellation 및 결과 보존을 검증합니다.

요청 단위 provider deadline

Layer / File(s)	Summary
공유 timeout budget contextual_orchestrator/orchestrator.py	Provider 호출, retry, backoff 및 failover가 하나의 요청 deadline을 사용합니다.
HTTP와 streaming 오류 처리 contextual_orchestrator/server.py	Deadline 헤더를 검증하고 만료 시 RequestDeadlineExceeded, HTTP 504 및 response.failed를 반환합니다.
신뢰성 검증 tests/test_provider_reliability.py, tests/test_orchestrated_responses_stream.py	Retry budget, backup provider, binary transport 및 deadline 오류 전파를 검증합니다.

Rust 및 런타임 경계

Layer / File(s)	Summary
Rust 토큰 패커 rust/token_counter/*, contextual_orchestrator/token_counting.py	cl100k packing, token 합산 및 weighted average를 Rust PyO3 확장으로 제공합니다.
빌드와 Compose 격리 Dockerfile, .github/workflows/tests.yml, compose.yaml	고정된 Maturin 빌드와 환경변수 기반 Compose 프로젝트 이름을 적용합니다.
가속기 런타임 문서 docs/adr/0006-native-accelerator-runtime-boundaries.md, docs/architecture.md, README.md	CPU PyO3, MLX, CUDA/OpenCL 및 Kubernetes의 런타임 경계를 기록합니다.

요청 계약과 운영 기록

Layer / File(s)	Summary
모델 및 요청 계약 contextual_orchestrator/__main__.py, contextual_orchestrator/server.py, tests/test_auto_discovery_server.py	자동 검색 capability와 reasoning effort 허용값을 갱신하고 명시적 model 검증을 추가합니다.
Budget 및 운영 기록 contextual_orchestrator/orchestrator.py, CHANGELOG.md, docs/product-technical-gap-baseline.md	모델별 budget meter, Decimal 비용 계산 및 변경 기록을 갱신합니다.

Estimated code review effort: 5 (Critical) | ~120 minutes

Merge Risk: 🟠 High · up to 2a181

This change routes remote embedding work through configured providers and adds durable batch execution behavior, but the current implementation can leave some jobs stuck, fail concurrent submissions, and turn caller timeouts into 500 errors and misleading provider-health records. These production-facing correctness and availability risks should be fixed before merge.

Sequence Diagram(s)

sequenceDiagram
  participant Client
  participant Server
  participant CostRoutingCoordinator
  participant ProviderEmbeddingBatchBackend
  participant Provider
  Client->>Server: 임베딩 배치 요청
  Server->>CostRoutingCoordinator: 입력과 요청 deadline 전달
  CostRoutingCoordinator->>ProviderEmbeddingBatchBackend: shard 작업 제출
  ProviderEmbeddingBatchBackend->>Provider: 임베딩 요청
  Provider-->>ProviderEmbeddingBatchBackend: 벡터와 prompt usage 반환
  ProviderEmbeddingBatchBackend-->>CostRoutingCoordinator: terminal 상태와 결과 저장
  CostRoutingCoordinator-->>Server: 결과와 provenance 반환
  Server-->>Client: HTTP 응답

Loading

sequenceDiagram
  participant Client
  participant Server
  participant Orchestrator
  participant ProviderRetry
  participant Provider
  Client->>Server: timeout header 포함 요청
  Server->>Orchestrator: monotonic deadline 전달
  Orchestrator->>ProviderRetry: 남은 budget으로 실행
  ProviderRetry->>Provider: 제한된 timeout으로 호출
  Provider-->>ProviderRetry: 응답 또는 오류
  ProviderRetry-->>Orchestrator: 성공 또는 RequestDeadlineExceeded
  Orchestrator-->>Server: 결과 또는 504 오류
  Server-->>Client: HTTP 응답

Loading

🚥 Pre-merge checks | ✅ 4 | ❌ 1

❌ Failed checks (1 warning)

Check name	Status	Explanation	Resolution
Docstring Coverage	⚠️ Warning	Docstring coverage is 40.38% which is insufficient. The required threshold is 80.00%. Docstring coverage is scoped to functions touched by this diff. Analyzed 312 functions across 37 files. (18 skippe…	Write docstrings for the functions missing them to satisfy the coverage threshold.

✅ Passed checks (4 passed)

Check name	Status	Explanation
Description Check	✅ Passed	Check skipped - CodeRabbit’s high-level summary is enabled.
Title check	✅ Passed	제목은 원격 에이전트에서 provider-backed embedding을 사용하도록 변경하는 PR의 핵심 내용을 정확하고 간결하게 설명합니다.
Linked Issues check	✅ Passed	Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check	✅ Passed	Check skipped because no linked issues were found for this pull request.

Full details: Docstring Coverage

Explanation

Docstring coverage is 40.38% which is insufficient. The required threshold is 80.00%. Docstring coverage is scoped to functions touched by this diff. Analyzed 312 functions across 37 files. (18 skipped: 17 unsupported, 1 too large.)

✨ Finishing Touches 💡 1

📝 Generate docstrings 💡

• Create stacked PR
• Commit on current branch

🧪 Generate unit tests (beta)

• Create PR with unit tests
• Commit unit tests in branch fix/provider-backed-embedding-batch

---

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

• X
• Mastodon
• Reddit
• LinkedIn

_{Comment @coderabbitai help to get the list of available commands.}