Skip to content

fix(api): accept the advertised gateway-default model on chat surfaces - #868

Open
seonghobae wants to merge 33 commits into
mainfrom
fix/gateway-default-chat-model
Open

fix(api): accept the advertised gateway-default model on chat surfaces#868
seonghobae wants to merge 33 commits into
mainfrom
fix/gateway-default-chat-model

Conversation

@seonghobae

@seonghobae seonghobae commented Aug 26, 2026

Copy link
Copy Markdown
Contributor

Problem

Model discovery and serving had drifted across gateway defaults, configured-gateway metadata, free routing, and provider privacy evidence.

Fix

  • Make omitted model, contextual-orchestrator, and orchestrator/auto one contract across Chat, Responses, and embeddings; keep orchestrator/free strictly zero-cost.
  • Discover configured gateway model, pricing, capability, and privacy metadata through KV credentials and an explicit HTTPS allowlist. Conflicting or partial deployment evidence stays unknown.
  • Preserve individual models and exclude non-chat capabilities only from chat routing.
  • Apply OpenRouter's official ZDR endpoint inventory and provider/endpoint data policies to paid and free models.
  • Persist privacy booleans and policy-source provenance; unknown remains distinct from unsupported.
  • Crawl official policy sources through Wardnet's bounded DNS-pinned fetch API. An explicitly ZDR-capable discovered model may enrich no-training/no-retention metadata only with a literal source quote.
  • Optionally render client-side policy pages through Camoufox MCP. Every tab receives Wardnet's dedicated-token proxy; deployment assigns Wardnet DNS, disables browser TRR, and blocks direct egress.
  • Fail over malformed responses only inside the selected bounded model group.

Official privacy sources:

Ecosystem dependencies

These remain stacked protected deliveries; their absence leaves policy enrichment unavailable rather than bypassing the boundary.

Validation

  • Latest complete local suite before the final concurrent hardening commit: 2383 passed.
  • Exact-head privacy/discovery/boundary/docstring suite: 75 passed.
  • Exact-head Ruff and diff checks pass.
  • Live configured-gateway and routing evidence remains recorded in the product gap baseline; no operator gateway hostname is committed or included here.
  • Exact-head hosted security/tests/reviews remain required before protected merge.

The /v1/models listing advertised 'contextual-orchestrator' as its first
entry and every batch request already defaulted to it, but
_require_pool_model special-cased only AUTO_MODEL/FREE_MODEL. The
gateway's own default id was therefore rejected with 400 on
/v1/chat/completions, /v1/completions, /v1/embeddings, and /v1/responses:
callers could submit async batch-routing jobs yet never hold a
conversation.

Introduce TaskOrchestrator.GATEWAY_DEFAULT_MODEL as the canonical virtual
id, resolve it with auto semantics in _require_pool_model (capability
callers still bind a concrete agent), and reference it at the remaining
server/orchestrator call sites so the advertised list and the accepted
set can no longer drift.
@coderabbitai

coderabbitai Bot commented Aug 26, 2026

Copy link
Copy Markdown

Warning

Review limit reached

Next included review available in 15 minutes.

View limit details

Limit details: You’ve used the included review currently available.

You've used all free OSS reviews for now. Wait for the free limit to reset to keep reviewing this public repository.

Learn how review limits work.

Review configuration:

⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Pro Plus

Run ID: d17e3e91-fdbc-4929-a7fa-5b5d1304eb15

📥 Commits

Reviewing files that changed from the base of the PR and between 56a898b and 6801f17.

⛔ Files ignored due to path filters (1)
  • uv.lock is excluded by !**/*.lock
📒 Files selected for processing (25)
  • README.md
  • contextual_orchestrator/__main__.py
  • contextual_orchestrator/model_discovery.py
  • contextual_orchestrator/orchestrator.py
  • contextual_orchestrator/privacy_policy_analysis.py
  • contextual_orchestrator/provider_bootstrap.py
  • contextual_orchestrator/provider_catalog_store.py
  • contextual_orchestrator/server.py
  • docs/kv-credentials.md
  • docs/planning/adrs/0032-model-group-cost-aware-discovery.md
  • docs/product-technical-gap-baseline.md
  • pyproject.toml
  • tests/test_auto_discovery_server.py
  • tests/test_chat_orchestration_mode_http_honesty.py
  • tests/test_discover_models_cli.py
  • tests/test_model_discovery.py
  • tests/test_model_discovery_boundaries.py
  • tests/test_model_strip_writeback_http_honesty.py
  • tests/test_orchestrated_responses_stream.py
  • tests/test_privacy_policy_analysis.py
  • tests/test_provider_bootstrap.py
  • tests/test_provider_catalog_store.py
  • tests/test_provider_reliability.py
  • tests/test_provider_tls.py
  • tests/test_responses_model_required_http_honesty.py

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

devin-ai-integration[bot]

This comment was marked as resolved.

@seonghobae
seonghobae enabled auto-merge August 26, 2026 09:05
devin-ai-integration[bot]

This comment was marked as resolved.

@devin-ai-integration devin-ai-integration Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Devin Review found 0 new potential issues.

Open in Devin Review

devin-ai-integration[bot]

This comment was marked as resolved.

devin-ai-integration[bot]

This comment was marked as resolved.

seonghobae and others added 2 commits August 26, 2026 03:20
* fix: discover configured gateway model catalog

* fix: require consensus for gateway pricing

* fix: normalize gateway bootstrap credential
devin-ai-integration[bot]

This comment was marked as resolved.

@seonghobae
seonghobae disabled auto-merge August 26, 2026 10:25
@seonghobae
seonghobae enabled auto-merge (squash) August 26, 2026 10:26
devin-ai-integration[bot]

This comment was marked as resolved.

devin-ai-integration[bot]

This comment was marked as resolved.

devin-ai-integration[bot]

This comment was marked as resolved.

devin-ai-integration[bot]

This comment was marked as resolved.

devin-ai-integration[bot]

This comment was marked as resolved.

devin-ai-integration[bot]

This comment was marked as resolved.

devin-ai-integration[bot]

This comment was marked as resolved.

devin-ai-integration[bot]

This comment was marked as resolved.

devin-ai-integration[bot]

This comment was marked as resolved.

@seonghobae
seonghobae disabled auto-merge August 26, 2026 12:37

@devin-ai-integration devin-ai-integration Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Devin Review found 1 new potential issue.

Open in Devin Review

Comment thread contextual_orchestrator/model_discovery.py
@seonghobae
seonghobae enabled auto-merge (squash) August 26, 2026 12:52
@seonghobae
seonghobae disabled auto-merge August 26, 2026 12:57
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant