BYOK: pluggable model providers (Gemini, Anthropic, OpenAI, xAI) — #29 plus review fixes - #34
Merged
Merged
Conversation
The AI layer only spoke Gemini: NewFromEnv demanded GEMINI_API_KEY, Generate
wrote the generateContent wire format inline, and both `explain` and `ask`
hardcoded "this sends data to Google" in their consent prompts. Anyone holding
an Anthropic, OpenAI, OpenRouter or xAI key — or running a model locally —
couldn't use it at all.
internal/ai now has a provider seam modelled on charmbracelet/fantasy's
Provider/LanguageModel pair, implemented over net/http rather than depending on
fantasy: it wraps the real vendor SDKs (anthropic-sdk-go, openai-go, genai,
aws-sdk-go-v2) and measured at 23 MB → ~65 MB of binary and 61 → ~170 modules,
for one non-streaming POST in an optional feature. Four providers, two wire
formats, zero new dependencies:
- gemini generateContent — unchanged behavior, default gemini-flash-latest
(a moving alias, so it tracks 3.7 Flash without a re-pin)
- anthropic /v1/messages, default claude-opus-5
- openai /chat/completions, default gpt-5.6-terra — also the compatibility
path for OpenRouter, Groq, Together, DeepSeek, xAI, Mistral and
every local runtime (Ollama, vLLM, LM Studio)
- xai /responses, default grok-4.6
Provider comes from PGBOT_AI_PROVIDER, else auto-detects from whichever key is
present (Gemini first, so existing setups are untouched). PGBOT_AI_MODEL,
PGBOT_AI_BASE_URL, PGBOT_AI_API_KEY and PGBOT_AI_REASONING_EFFORT override;
PGBOT_GEMINI_MODEL/URL still work. Keys are still read only from the
environment, never a flag — now enforced in one place for every provider.
Wire-format details that are load-bearing, each found by probing the live APIs:
- Anthropic rejects `temperature` on current models (400), so the provider never
sends it — Call.Temperature is a hint, not a contract. A refusal is an HTTP 200
with empty content, so stop_reason is checked before reading blocks.
- OpenAI reasoning models reject `max_tokens` ("use max_completion_tokens") and
take reasoning_effort; their cap covers hidden reasoning as well as the answer,
so it is floored at 32k — 8192 returns empty text with finish_reason "length".
Dispatch is by model id, so local runtimes keep the plain shape.
- The Responses API defaults `store` to true, retaining the findings server-side
after the call. pgbot always sends store:false — that retention isn't the
disclosure the user consented to.
- xAI returns errors as {"code":…,"error":"<string>"} where OpenAI nests an
object. Both providers now decode either shape, or the real message is lost.
`explain` and `ask` name the actual destination before sending ("…to xai at
api.x.ai (model grok-4.6)"), and with a local endpoint say nothing leaves the
machine and skip the prompt entirely. The model call also gets its own deadline
instead of the leftovers of the 45s collection budget, which a CPU-bound local
model always blew.
Verified end-to-end against live endpoints: Grok 4.6 via /responses and a local
Ollama via /chat/completions both produced real explanations for `explain` and
`ask`; Anthropic reached auth and surfaced its error; bad-key paths degrade to
the deterministic report with the vendor's message intact. Unit tests cover each
provider's request shape, error handling and env precedence. Binary unchanged at
16 MB, still 61 modules.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
… the docs Review follow-ups on the BYOK provider layer, each pinned by a test: - An explicit PGBOT_AI_PROVIDER=xai/grok no longer borrows OPENAI_API_KEY; only the endpoint-shaped `responses` alias means "the Responses API at OpenAI". With PGBOT_AI_BASE_URL also set the fallback sent OpenAI's key to api.x.ai. - The openai-family aliases name their endpoint: `openrouter` goes to OpenRouter even when the key arrives as PGBOT_AI_API_KEY (the raw-variable check sent it to api.openai.com), `ollama` to localhost:11434, and `openai-compatible` asks for PGBOT_AI_BASE_URL instead of guessing OpenAI. - A remote PGBOT_AI_BASE_URL with no PGBOT_AI_PROVIDER is refused. Auto-detection would pick a wire format and key from whatever vendor variable is in the shell and send that key to the named host — silently under --yes. Local endpoints keep working unnamed. - Anthropic and Responses requests floor the output cap (16k / 32k): both families think inside max_tokens, and the 8192 hint that suits a plain chat model left the explanation truncated or empty, as the OpenAI path already handled with reasoningTokenFloor. - A plain-http remote endpoint gets a cleartext warning on the consent line. - README: quickstart, environment table, --yes text, and the privacy section still described the two-provider world; CHANGELOG entries, including the OpenAI default moving from gpt-4o-mini to gpt-5.6-terra at xhigh effort. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_013qGZKWgfGTBCoHsDjy1SuB
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Carries #29 by @edwardsb unchanged (commit 0a8ade2), merged with current main, plus the review follow-up commit 2c48f04 described in #29 (comment).
The contributor's fork is organization-owned, so the follow-up could not be pushed to #29's branch; this PR exists to run CI on the combined tree and land it. Merging it merges #29's commit, and GitHub marks #29 as merged.
Closes #29. Addresses #32 (an OpenAI-compatible endpoint is configured with
PGBOT_AI_PROVIDER=openai+PGBOT_AI_BASE_URL+PGBOT_AI_API_KEY; no vendor-specific case needed).🤖 Generated with Claude Code
https://claude.ai/code/session_013qGZKWgfGTBCoHsDjy1SuB