Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
7 changes: 7 additions & 0 deletions .env.template
Original file line number Diff line number Diff line change
Expand Up @@ -329,6 +329,13 @@
# (e.g. /v1/chat/completions and /v1/responses items).
# ENABLE_GUARDRAILS_FOR_BATCH_PROCESSING=false

# Enable/disable outbound header policies globally (default: true)
# HEADER_POLICIES_ENABLED=true

# Replace header_policies.policies with a JSON array. Prefer config.yaml for
# readability and ${VARIABLE} expansion for literal secret values.
# HEADER_POLICIES_JSON=[{"name":"pin-beta","step":20,"methods":["POST"],"paths":["/v1/chat/completions"],"actions":[{"action":"set","header":"Anthropic-Beta","value":"context-1m-2025-08-07"}]}]

# In-memory buffer size before flushing to storage (default: 1000)
# USAGE_BUFFER_SIZE=1000

Expand Down
3 changes: 2 additions & 1 deletion CLAUDE.md
Original file line number Diff line number Diff line change
Expand Up @@ -132,7 +132,8 @@ Full reference: `.env.template` and `config/config.yaml`
- **HTTP client:** `HTTP_TIMEOUT` (600s), `HTTP_RESPONSE_HEADER_TIMEOUT` (600s); also settable via the `http:` block in `config.yaml` (env vars win)
- **Resilience:** Configured via `config/config.yaml` - global `resilience.retry.*` and `resilience.circuit_breaker.*` defaults with optional per-provider overrides under `providers.<name>.resilience.retry.*` and `providers.<name>.resilience.circuit_breaker.*`. Retry defaults: `max_retries` (3), `initial_backoff` (1s), `max_backoff` (30s), `backoff_factor` (2.0), `jitter_factor` (0.1). Circuit breaker defaults: `failure_threshold` (5), `success_threshold` (2), `timeout` (30s). Breaker state is per-process and exported as the `gomodel_circuit_breaker_state` gauge when metrics are enabled. The dashboard's provider status also folds in real-traffic request health: each provider's runtime row carries `request_health` (last observed breaker state plus per-model request/error counts over a 10-minute sliding window; a model with ≥3 errors and a ≥50% error rate is flagged). An open breaker turns the provider card's status pill to "Circuit Open" (unhealthy), a half-open breaker or a flagged model degrades an otherwise healthy provider ("Recovering"/"Degraded"), and the expanded card details list recent per-model traffic with the latest error. Signals only worsen the discovery-based status, never improve it; tracking is in-memory per instance, and providers with no recent requests show discovery-based status only.
- **Metrics:** `METRICS_ENABLED` (false), `METRICS_ENDPOINT` (/metrics)
- **Guardrails:** Definitions are persisted in the `guardrail_definitions` store and managed via the admin API/dashboard; `config/config.yaml` entries are validated and upserted into that store at startup (a seed, not the source of truth). `GUARDRAILS_ENABLED` env var gates the feature.
- **Guardrails:** Message guardrail definitions are managed through the Guardrails dashboard and `/admin/guardrails`; `config/config.yaml` entries seed the store at startup. `GUARDRAILS_ENABLED` gates the feature. Types: `system_prompt` and `llm_based_altering`.
- **Header policies:** Outbound provider-request header policies are a separate egress subsystem with dedicated `header_policy_definitions` persistence, managed through the Header Policies dashboard and `/admin/header-policies`. Declare them under `header_policies.policies` or `HEADER_POLICIES_JSON`; `HEADER_POLICIES_ENABLED` is the independent kill switch. Conditions support `matches`/`equals`/`present`; actions support `set` with `value` or `from_header`, and `remove`. Credential, transport, and payload-metadata headers are rejected. Policies apply on translated, passthrough, realtime, WebRTC, and MCP upstream requests, participate in response-cache identity, and record redacted/name-only deltas on audit `request_revisions`; values copied from credential-like source headers stay redacted under innocuous target names. Realtime/MCP workflow matching is user-path/global only, so constrain provider-specific rules with `paths`. Historical `header_modification` guardrail rows are migrated once to the dedicated store and legacy workflow references remain readable.
- **Provider API key rotation:** Any API-key provider accepts several keys: `<PROVIDER>[_SUFFIX]_API_KEY_<n>` env vars (numbered from 2; `_1` is accepted as a synonym for the unsuffixed key) or `providers.<name>.api_keys` in `config.yaml` (merged after `api_key`, de-duplicated, unresolved `${...}` entries dropped; env replaces the whole YAML list). Two or more keys turn on round-robin rotation, drawn per outbound HTTP request — including retries, so a 429'd request retries under the next key. Realtime websocket sessions pick a key per session. Counters are in-memory per instance. The trailing number names a key, not a provider: `OPENAI_API_KEY_2` is key 2 of `openai`, while `OPENAI_REGION_2_API_KEY` is the sole key of provider `openai-region-2`. **Rotation defeats provider prompt caching** (providers scope the cache to the key that filled it); use it to lift per-key rate limits, not to save cost. Keyless (Ollama, vLLM) and non-API-key providers (Vertex, Bedrock) are unaffected.
- **Providers:** `OPENAI_API_KEY`, `ANTHROPIC_API_KEY`, `ANTHROPIC_DEFAULT_MAX_TOKENS` (optional default `max_tokens` for Anthropic-translated requests that omit it; default 4096), `GEMINI_API_KEY`, `USE_GOOGLE_GEMINI_NATIVE_API` (true by default; false uses Gemini's OpenAI-compatible chat API), `XAI_API_KEY`, `GROQ_API_KEY`, `FIREWORKS_API_KEY`, `FIREWORKS_BASE_URL` (optional Fireworks AI endpoint override; default `https://api.fireworks.ai/inference/v1`), `META_API_KEY`, `META_BASE_URL` (optional Meta Model API endpoint override; default `https://api.meta.ai/v1`; Muse Spark models, e.g. `muse-spark-1.1`), `OPENROUTER_API_KEY`, `OPENROUTER_SITE_URL`/`OPENROUTER_APP_NAME` (optional OpenRouter attribution headers), `ZAI_API_KEY`, `ZAI_BASE_URL` (optional Z.ai endpoint override), `MINIMAX_API_KEY`, `MINIMAX_BASE_URL` (optional MiniMax endpoint override), `XIAOMI_API_KEY`, `XIAOMI_BASE_URL` (optional Xiaomi MiMo endpoint override), `OPENCODE_GO_API_KEY`, `OPENCODE_GO_BASE_URL` (optional OpenCode Go/Zen endpoint override; default `https://opencode.ai/zen/go/v1`), `OPENCODE_GO_MESSAGES_MODELS` (optional comma-separated model IDs routed to the Anthropic-native `/messages` endpoint instead of `/chat/completions`; default `qwen3.7-max`), `BAILIAN_API_KEY`, `BAILIAN_BASE_URL` (optional Bailian base URL for region switching; default `https://dashscope.aliyuncs.com/compatible-mode/v1`), `AZURE_API_KEY`, `AZURE_BASE_URL` (Azure OpenAI deployment base URL), `AZURE_API_VERSION` (optional Azure API version), `ORACLE_API_KEY` (Oracle API key), `ORACLE_BASE_URL` (Oracle OpenAI-compatible base URL), `BEDROCK_BASE_URL` (Bedrock Runtime region or endpoint), `BEDROCK_MANTLE_API_KEY`, `BEDROCK_MANTLE_BASE_URL` (Mantle region or endpoint), `BEDROCK_MANTLE_API_MODE` (`auto`, `openai`, or `standard`), `<PROVIDER>[_SUFFIX]_MODELS` (comma-separated configured model list for any provider type), `OLLAMA_BASE_URL`, `VLLM_BASE_URL`, `VLLM_API_KEY` (optional upstream vLLM bearer token)
- **Provider model metadata:** `providers.<name>.models` accepts either model IDs (strings) or `{id, metadata}` objects. When `metadata` is supplied (`display_name`, `context_window`, `max_output_tokens`, `modes`, `capabilities`, `pricing`, …) it is merged onto the remote ai-model-list entry during enrichment, with operator values winning per-field. Primary use case: advertising context windows, capabilities, and pricing for local models (Ollama) and other custom endpoints whose IDs are not in the upstream registry.
51 changes: 40 additions & 11 deletions cmd/gomodel/docs/docs.go

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

24 changes: 24 additions & 0 deletions config/config.example.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -279,6 +279,30 @@ guardrails:
# skip_content_prefix: "### safe"
# # prompt: "Custom rewrite instructions here."

header_policies:
enabled: true # env: HEADER_POLICIES_ENABLED; default true
policies:
# Named outbound policies are bound to the managed default workflow by step.
# Conditions all have to match; actions run in order. Credential, transport,
# content-type, and content-encoding headers cannot be read or changed.
# - name: "pin-anthropic-beta"
# description: "Enable the long-context Anthropic beta for Cline clients"
# step: 20
# methods: ["POST"] # optional; empty matches every method
# paths: ["/v1/chat/completions", "/p/anthropic/*"] # trailing * matches a prefix
# when:
# - header: "User-Agent"
# matches: "^cline/" # RE2 regex; or use equals/present
# actions:
# - action: "set"
# header: "Anthropic-Beta"
# value: "context-1m-2025-08-07"
# - action: "set"
# header: "X-Team"
# from_header: "X-Client-Team"
# - action: "remove"
# header: "X-Internal-Debug"

failover:
enabled: true # env: FAILOVER_ENABLED; default true
manual_rules_path: "config/failover.example.json" # optional JSON map: {"primary-model": ["fallback-1", "provider/model"]}
Expand Down
Loading