BYOK: pluggable model providers (Gemini, Anthropic, OpenAI, xAI)

[edwardsb]

Summary

Expand the existing OpenAI and Gemini integration to support Anthropic, xAI, and OpenAI-compatible services. The implementation uses net/http and adds no dependencies.

Providers

Provider	Default model	API
Gemini	gemini-flash-latest	generateContent
Anthropic	claude-opus-5	/v1/messages
OpenAI	gpt-5.6-terra	/chat/completions
xAI	grok-4.6	/responses

The OpenAI-compatible provider also supports OpenRouter, Groq, Together, DeepSeek, Mistral, Ollama, vLLM, and LM Studio.

Configuration

PGBOT_AI_PROVIDER selects a provider. If unset, pgbot detects one from the available API keys. OpenAI remains first in the detection order to preserve current behavior.

General overrides:

• PGBOT_AI_MODEL
• PGBOT_AI_BASE_URL
• PGBOT_AI_API_KEY
• PGBOT_AI_REASONING_EFFORT

Existing PGBOT_OPENAI_MODEL, PGBOT_OPENAI_URL, PGBOT_GEMINI_MODEL, and PGBOT_GEMINI_URL settings remain supported. Keys are read only from environment variables.

Behavior

• Consent prompts identify the provider, host, and model receiving the findings.
• Local endpoints do not require confirmation.
• Model requests have a separate three-minute timeout from database collection.
• AI failures do not suppress the deterministic report.
• The Responses API sends store: false.
• Anthropic requests omit temperature, which current models reject.
• OpenAI reasoning models use max_completion_tokens and reasoning_effort.
• OpenAI-compatible services continue to use /chat/completions.

The provider interface follows the Provider and LanguageModel structure from charmbracelet/fantasy, implemented directly over net/http.

Verification

Live end-to-end tests against PostgreSQL passed after rebasing onto current upstream main:

• Gemini: gemini-flash-latest
• Anthropic: claude-opus-5
• OpenAI: gpt-5.6-terra
• xAI: grok-4.6 via /responses

Ollama was also verified through /chat/completions, including local-endpoint consent behavior.

The repository gate passes: build, vet, golangci-lint, all Go tests, and Linux/macOS builds for amd64 and arm64.

[edwardsb]

@alexshapalov hey Alex, not sure if you're accepting PRs, but I thought it would be cool to give folks more options on inference providers.

I tested the ones listed in the description. I also recently tested Muse 1.3 from Meta using the OpenAI configuration (muse is openai compatible).

If this isn't something you'd like to maintain, feel free to close the PR, nbd

[alexshapalov]

Code review

Found 5 issues, all fixed in 2c48f04 (pushed to this branch with maintainer edits, on top of a merge of main):

1. An explicit PGBOT_AI_PROVIDER=xai (or grok) with no xAI key silently borrowed OPENAI_API_KEY; with PGBOT_AI_BASE_URL also set, that sent OpenAI's key to api.x.ai. Only the endpoint-shaped responses alias now falls back to the OpenAI key.

pgbot/internal/ai/resolve.go

Lines 71 to 83 in 0a8ade2

	label := "xai"
	if key == "" {
	key = firstEnv("XAI_API_KEY", "GROK_API_KEY")
	}
	if key == "" && firstEnv("OPENAI_API_KEY") != "" {
	key, label = firstEnv("OPENAI_API_KEY"), "openai"
	if base == "" {
	base = envOr("PGBOT_OPENAI_URL", defaultOpenAIURL)
	}
	if model == "" {
	model = envOr("PGBOT_OPENAI_MODEL", defaultOpenAIModel)
	}
	}

2. The openai-family aliases picked their endpoint from the raw key variables, not the alias: PGBOT_AI_PROVIDER=openrouter with the key given as PGBOT_AI_API_KEY went to api.openai.com, and ollama with no base URL went to OpenAI too. Each alias now names its endpoint, and openai-compatible asks for PGBOT_AI_BASE_URL.

pgbot/internal/ai/resolve.go

Lines 101 to 109 in 0a8ade2

	if base == "" {
	base = envOr("PGBOT_OPENAI_URL", "")
	if base == "" {
	base = defaultOpenAIURL
	if os.Getenv("OPENAI_API_KEY") == "" && os.Getenv("OPENROUTER_API_KEY") != "" {
	base = "https://openrouter.ai/api/v1"
	}
	}
	}

3. A remote PGBOT_AI_BASE_URL with no PGBOT_AI_PROVIDER let auto-detection choose the wire format and key from whatever vendor variable was in the shell (an ANTHROPIC_API_KEY left by another tool) and send that key to the named host, with nothing shown under --yes. It is now refused with a message asking for the provider; a local base URL still works unnamed.

pgbot/internal/ai/resolve.go

Lines 28 to 35 in 0a8ade2

	func Resolve() (LanguageModel, error) {
	name := strings.ToLower(envOr("PGBOT_AI_PROVIDER", ""))
	if name == "" {
	var err error
	if name, err = detectProvider(); err != nil {
	return nil, err
	}
	}

4. The Anthropic and Responses paths forwarded the 8192-token hint as the output cap even though Opus 5 and grok-4 think inside that same budget; the /chat/completions path already floors reasoning models at 32k for exactly this reason. Both now floor it (16k Anthropic, 32k Responses).

pgbot/internal/ai/anthropic.go

Lines 84 to 88 in 0a8ade2

	func (m *anthropicModel) Generate(ctx context.Context, c Call) (*Response, error) {
	maxTokens := 8192
	if c.MaxOutputTokens != nil {
	maxTokens = int(*c.MaxOutputTokens)
	}

5. The README was only partly updated: the quickstart still offered OpenAI (gpt-4o-mini) or Gemini, the environment table listed only the old keys and said PGBOT_AI_PROVIDER forces openai or gemini, and the privacy section still named two providers.

pgbot/README.md

Lines 62 to 68 in 0a8ade2

	```sh
	export DATABASE_URL="postgres://pgbot_ro:…@host:5432/db?sslmode=require"
	# optional, for `pgbot ask` / `pgbot explain` — one of:
	export OPENAI_API_KEY=sk-… # → OpenAI (gpt-4o-mini by default)
	export GEMINI_API_KEY=… # → Google Gemini (AI Studio key)
	```

Also added a cleartext warning on the consent line for a plain-http remote endpoint, and CHANGELOG entries.

One thing for the maintainer rather than a defect: the OpenAI default moves from gpt-4o-mini to gpt-5.6-terra, classed as a reasoning model and sent with reasoning_effort: xhigh and a 32k completion cap. The commit that introduced the OpenAI provider chose a small cheap model on purpose ("a small, cheap chat model is plenty for an explanation task"). Anyone with only OPENAI_API_KEY set gets the new cost and latency with no config change. Left as submitted and documented in the changelog with the override; it is a one-constant change if you want the cheaper default.

Checked for bugs, consent and key handling, git history, prior PRs and issues (#32 is satisfied by this design without a vendor-specific case), and code-comment guidance; this repo has no CLAUDE.md. Verified the local-endpoint check fails safe against spoofed hostnames, keys travel only in headers, and response bodies are bounded.

🤖 Generated with Claude Code

_{- If this code review was useful, please react with 👍. Otherwise, react with 👎.}

[alexshapalov]

Correction to the review above: this fork is organization-owned, so GitHub refused the maintainer push. The follow-up commit 2c48f04 lives in #34, which carries your commit 0a8ade2 unchanged plus the fixes; merging #34 lands this PR's commit and marks it merged. Thanks for the contribution.