Problem
Unstract currently supports a fixed set of LLM providers, which makes it difficult to use endpoints that are OpenAI-compatible but are not the official OpenAI service.
Examples include:
- private / enterprise gateways exposing an OpenAI-style chat API
- OpenRouter
- OpenWebUI-generated OpenAI-compatible endpoints
- internal or regional deployments that require a custom base URL
There is already user demand for this direction:
In #856, a maintainer also suggested adding another adapter similar to the existing ones for a private/custom LLM setup.
Proposal
Add a dedicated OpenAI Compatible LLM adapter instead of broadening the existing OpenAI adapter.
The goal is to support LLM endpoints that follow the OpenAI chat-completions style API but are hosted behind a different provider or gateway.
Why a new adapter instead of changing the existing OpenAI adapter?
Using a separate adapter keeps the current behavior explicit and avoids mixing two different concepts:
- official OpenAI
- OpenAI-compatible but non-OpenAI endpoints
This also keeps the support boundary clearer and reduces regression risk for existing OpenAI users.
Initial scope
For an initial PR, I propose keeping the scope intentionally small:
- LLM only
- no embedding changes
- no x2text / OCR changes
- keep the existing
OpenAI adapter unchanged
A minimal first version would support the common configuration needed for compatible endpoints:
If maintainers prefer, provider-specific auth/config extensions can be handled in follow-up PRs instead of the first change.
Pricing / usage semantics
I am not proposing to change pricing semantics in the first PR.
For custom or OpenAI-compatible providers that do not have pricing metadata available, usage can still be recorded, while cost calculation can remain best-effort / unavailable until provider-model mapping is added separately.
Why this would help
This would make Unstract easier to adopt in environments where users already expose LLMs through a compatible gateway, without weakening the semantics of the existing provider adapters.
If this direction sounds acceptable, I can work on a small PR that introduces the new adapter with LLM-only support first.
Problem
Unstract currently supports a fixed set of LLM providers, which makes it difficult to use endpoints that are OpenAI-compatible but are not the official OpenAI service.
Examples include:
There is already user demand for this direction:
In #856, a maintainer also suggested adding another adapter similar to the existing ones for a private/custom LLM setup.
Proposal
Add a dedicated
OpenAI Compatible LLMadapter instead of broadening the existingOpenAIadapter.The goal is to support LLM endpoints that follow the OpenAI chat-completions style API but are hosted behind a different provider or gateway.
Why a new adapter instead of changing the existing OpenAI adapter?
Using a separate adapter keeps the current behavior explicit and avoids mixing two different concepts:
This also keeps the support boundary clearer and reduces regression risk for existing OpenAI users.
Initial scope
For an initial PR, I propose keeping the scope intentionally small:
OpenAIadapter unchangedA minimal first version would support the common configuration needed for compatible endpoints:
api_baseapi_keymodelIf maintainers prefer, provider-specific auth/config extensions can be handled in follow-up PRs instead of the first change.
Pricing / usage semantics
I am not proposing to change pricing semantics in the first PR.
For custom or OpenAI-compatible providers that do not have pricing metadata available, usage can still be recorded, while cost calculation can remain best-effort / unavailable until provider-model mapping is added separately.
Why this would help
This would make Unstract easier to adopt in environments where users already expose LLMs through a compatible gateway, without weakening the semantics of the existing provider adapters.
If this direction sounds acceptable, I can work on a small PR that introduces the new adapter with LLM-only support first.