Skip to content

feat(inference): support workload-specific model endpoints - #1380

Open
PsiACE wants to merge 4 commits into
masterfrom
codex/feat/custom-model-endpoints
Open

feat(inference): support workload-specific model endpoints#1380
PsiACE wants to merge 4 commits into
masterfrom
codex/feat/custom-model-endpoints

Conversation

@PsiACE

@PsiACE PsiACE commented Aug 27, 2026

Copy link
Copy Markdown
Member

Which issue or RFC does this PR close?

Closes #1379.

Rationale for this change

Generation, embeddings, and LLM reranking may use different model services and require different routing or authentication headers.

What changes are included in this PR?

  • Add workload-specific models, base URLs, static headers, and model settings.
  • Configure sensitive headers on provider clients instead of request settings.
  • Keep LLM reranking on the regular model interface.
  • Document provider-owned operation paths and configuration limits.
  • Add acceptance coverage for separate generation, embedding, and rerank services.

Are there any user-facing changes?

New inference environment variables are available. Existing configurations keep their current behavior.

Custom endpoints and headers require an explicit OpenAI- or Anthropic-compatible model identifier. Arbitrary operation-path rewriting is not supported.

How was this change tested?

  • make check
  • make test
  • make docs-test
  • Local acceptance services for generation, embeddings, and LLM reranking

AI usage statement

OpenAI Codex with GPT-5 was used for implementation and review.

Comment thread src/powercontext/builtin/runtime/composition.py Outdated
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

feat: support workload-specific inference endpoints

2 participants