From e03f823b31a10ae9f5a93268468ea938fe7fcd29 Mon Sep 17 00:00:00 2001 From: tomek-labuk Date: Tue, 4 Aug 2026 14:27:50 +0200 Subject: [PATCH] Align docs with 0.7.0 API spec --- app/_ai_gateway_entities/ai-agent.md | 28 ++++++++++----- .../ai-identity-provider.md | 35 +++++++++++++------ app/_ai_gateway_entities/ai-model.md | 32 +++++++++++------ .../ai-gateway/get-started-with-ai-gateway.md | 8 +++-- ...e-claude-code-with-ai-gateway-anthropic.md | 6 ++-- .../use-claude-code-with-ai-gateway-azure.md | 2 +- ...use-claude-code-with-ai-gateway-bedrock.md | 6 ++-- ...e-claude-code-with-ai-gateway-dashscope.md | 6 ++-- .../use-claude-code-with-ai-gateway-gemini.md | 6 ++-- ...claude-code-with-ai-gateway-huggingface.md | 6 ++-- .../use-claude-code-with-ai-gateway-openai.md | 6 ++-- .../use-claude-code-with-ai-gateway-vertex.md | 2 +- .../ai-gateway/use-codex-with-ai-gateway.md | 6 ++-- app/ai-gateway/configure-on-prem.md | 9 +++-- app/ai-gateway/load-balancing.md | 6 ++-- 15 files changed, 108 insertions(+), 56 deletions(-) diff --git a/app/_ai_gateway_entities/ai-agent.md b/app/_ai_gateway_entities/ai-agent.md index 9344ee3e533..1d5394c2a4d 100644 --- a/app/_ai_gateway_entities/ai-agent.md +++ b/app/_ai_gateway_entities/ai-agent.md @@ -27,6 +27,8 @@ related_resources: url: /ai-gateway/entities/ - text: AI Policy entity url: /ai-gateway/entities/ai-policy/ + - text: AI Identity Provider entity + url: /ai-gateway/entities/ai-identity-provider/ - text: AI Consumer Group entity url: /ai-gateway/entities/ai-consumer-group/ - text: A2A protocol specification @@ -65,6 +67,13 @@ faqs: Set the [`access.acls`](#schema-aigateway-agent-access) field on the AI Agent with an allow list or a deny list. Each entry is a string that references an AI Consumer, AI Consumer Group, or Authenticated Group by name. + - q: How do I authenticate requests to an AI Agent? + a: | + Reference an [AI Identity Provider](/ai-gateway/entities/ai-identity-provider/) by name or id in the AI Agent's + [`access.identity_providers`](#schema-aigateway-agent-access) array, the same field used on AI Models. An AI Agent + currently accepts up to one AI Identity Provider reference. Attaching an authentication AI Policy directly to an + AI Agent's `policies` field isn't supported. + - q: How do I attach AI Policies to an AI Agent? a: | Configuration that applies to the AI Agent goes through the [AI Policy entity](/ai-gateway/entities/ai-policy/). @@ -140,7 +149,7 @@ rows: When an Agent has type `a2a`, proxied traffic is processed in four phases: -1. **Access**. Detects whether the request is an A2A operation (JSON-RPC or REST binding). When statistics logging is enabled, this starts an OpenTelemetry span and records the request body for payload logging if that's also enabled. +1. **Access**. Detects whether the request is an A2A operation (JSON-RPC or REST binding). This starts an OpenTelemetry span, and records the request body for payload logging if that's enabled. 1. **Header filter**. Detects streaming responses (`Content-Type: text/event-stream`) and records time to first byte. Buffers agent-card responses for URL rewriting. 1. **Body filter**. Streams SSE chunks through to the client without buffering. Buffers non-streaming responses to extract task metadata. Rewrites agent-card URLs to the gateway address. Emits analytics at end of response. 1. **Log**. Finalizes the OpenTelemetry span with task state, task ID, and any error information. @@ -282,14 +291,14 @@ When an upstream agent returns an agent card, the runtime rewrites the [`url`](# ## Logging and observability -To track agent performance, debug issues, and monitor A2A traffic patterns, enable statistics logging. {{site.ai_gateway}} emits structured A2A telemetry that flows to {{site.konnect_short_name}} analytics, logging plugins, and OpenTelemetry for full visibility into agent operations. +For `a2a` type Agents, {{site.ai_gateway}} automatically emits structured A2A telemetry to track agent performance, debug issues, and monitor A2A traffic patterns. This telemetry flows to {{site.konnect_short_name}} analytics, logging plugins, and OpenTelemetry for full visibility into agent operations, with no separate toggle required. The telemetry data is emitted into the `ai.a2a` namespace (consumed by {{site.konnect_short_name}} analytics and logging AI Policies) and creates a `kong.a2a` child span when you've configured [{{site.base_gateway}} tracing](/gateway/tracing/). For the canonical metric and attribute list, see [A2A metrics](/ai-gateway/ai-otel-metrics/#a2a-metrics). {:.info} -> When statistics logging is enabled, the runtime removes the `Accept-Encoding` request header -> before forwarding to the upstream. This prevents compressed responses that the runtime can't -> parse for metadata extraction. +> For `a2a` type Agents, the runtime removes the `Accept-Encoding` request header before forwarding +> to the upstream. This prevents compressed responses that the runtime can't parse for metadata +> extraction. Payload logging additionally captures request and response bodies. Payloads are truncated at the configured payload size limit. @@ -305,7 +314,7 @@ You can view A2A analytics in {{site.konnect_short_name}} Explorer and Dashboard ### OpenTelemetry span attributes -When statistics logging is enabled and {{site.base_gateway}} tracing is configured, the runtime creates a `kong.a2a` child span with the following attributes: +For `a2a` type Agents, when {{site.base_gateway}} tracing is configured, the runtime creates a `kong.a2a` child span with the following attributes: {% include md/ai-gateway/v2/otel-span-attributes.md %} @@ -313,7 +322,9 @@ When statistics logging is enabled and {{site.base_gateway}} tracing is configur To restrict which AI Consumers or teams can reach a specific agent, use ACLs. The [`access.acls`](#schema-aigateway-agent-access) field defines either an `allow` or a `deny` list of identities that can access the agent. Each entry references an [AI Consumer](/ai-gateway/entities/ai-consumer/), [AI Consumer Group](/ai-gateway/entities/ai-consumer-group/), or Authenticated Group by name. An Authenticated Group is a dynamic group representing all consumers authenticated via a specific OAuth2 scope or claim. Access is enforced before traffic reaches the upstream agent. -For per-request authentication and identity validation, attach an authentication AI Policy to the AI Agent. +For per-request authentication and identity validation, reference an [AI Identity Provider](/ai-gateway/entities/ai-identity-provider/) in the [`access.identity_providers`](#schema-aigateway-agent-access) array, the same way you would for an [AI Model](/ai-gateway/entities/ai-model/#access-control). Attaching an authentication AI Policy directly to the AI Agent's `policies` field isn't supported; authentication for AI Agents is configured exclusively through AI Identity Providers. + +An AI Agent currently accepts up to one AI Identity Provider reference. ACLs are evaluated only after the AI Consumer's identity is resolved through this authentication step. ## Attach AI Policies @@ -326,7 +337,7 @@ For available policy types and configuration, see the [AI Policy entity](/ai-gat Before creating an AI Agent with access restrictions, create an AI Consumer Group to reference in [`access.acls`](#schema-aigateway-agent-access). This example references a group named `internal-teams`. See [Set up an AI Consumer Group](/ai-gateway/entities/ai-consumer-group/#set-up-an-ai-consumer-group) to create it, or substitute the name of your own AI Consumer, AI Consumer Group, or Authenticated Group in `access.acls.allow`. -The following example creates an `a2a` Agent that proxies traffic to an upstream A2A agent at `https://booking-agent.internal.kongair.com`, with statistics logging enabled and access restricted to the `internal-teams` Consumer Group. +The following example creates an `a2a` Agent that proxies traffic to an upstream A2A agent at `https://booking-agent.internal.kongair.com`, with access restricted to the `internal-teams` Consumer Group. {:.info} > This example proxies to a placeholder upstream at `https://booking-agent.internal.kongair.com`. Substitute the URL of your own running A2A agent in [`config.url`](#schema-aigateway-agent-config-url). Because this Agent has `type: a2a`, requests must use the A2A JSON-RPC envelope (`jsonrpc: "2.0"`, `id`, `method: "message/send"`, `params.message` with `kind` and `messageId`). A flat `{"message": {...}}` body without that envelope is rejected by the upstream agent itself (for example, `"Invalid Request: jsonrpc must be 2.0"`), not by {{site.ai_gateway}}. @@ -348,7 +359,6 @@ data: paths: - /kongair-flight-booking logging: - statistics: true payloads: false max_payload_size: 1048576 {% endentity_example %} diff --git a/app/_ai_gateway_entities/ai-identity-provider.md b/app/_ai_gateway_entities/ai-identity-provider.md index 1fc98e7739e..cf4672bfb2e 100644 --- a/app/_ai_gateway_entities/ai-identity-provider.md +++ b/app/_ai_gateway_entities/ai-identity-provider.md @@ -11,7 +11,7 @@ permalink: /ai-gateway/entities/ai-identity-provider/ breadcrumbs: - /ai-gateway/ - /ai-gateway/entities/ -description: Configure inbound AI Consumer authentication for AI Models in {{site.ai_gateway}}. +description: Configure inbound AI Consumer authentication for AI Models and AI Agents in {{site.ai_gateway}}. schema: api: konnect/ai-gateway path: /schemas/AIGatewayIdentityProvider @@ -27,6 +27,8 @@ related_resources: url: /ai-gateway/entities/ai-model/ - text: AI Model Provider entity url: /ai-gateway/entities/ai-model-provider/ + - text: AI Agent entity + url: /ai-gateway/entities/ai-agent/ - text: AI Consumer entity url: /ai-gateway/entities/ai-consumer/ - text: AI Consumer Group entity @@ -48,16 +50,23 @@ faqs: AI Identity Provider simultaneously. An AI Consumer's request is authenticated if it satisfies either provider. + - q: Can an AI Agent use an AI Identity Provider too? + a: | + Yes. Reference an AI Identity Provider by `name` or `id` in the AI Agent's `access.identity_providers` + array, the same field used on AI Models. An AI Agent currently accepts up to one AI Identity Provider + reference. Attaching an authentication AI Policy directly to an AI Agent's `policies` field isn't + supported; AI Identity Providers are the only supported way to authenticate AI Agent traffic. + - q: What happens when a request carries no valid credentials? a: | {{site.ai_gateway}} treats the request as an anonymous AI Consumer. A request-termination policy on that anonymous AI Consumer returns `401 Unauthorized` before the request reaches - the AI Model. + the AI Model or AI Agent. - - q: Can I reuse the same AI Identity Provider across multiple AI Models? + - q: Can I reuse the same AI Identity Provider across multiple AI Models or AI Agents? a: | Yes. Create an AI Identity Provider once and reference it by `name` or `id` in the - `access.identity_providers` array of any AI Model in the same gateway. + `access.identity_providers` array of any AI Model or AI Agent in the same gateway. - q: Which OIDC flows does the openid-connect type support? a: | @@ -69,14 +78,14 @@ faqs: ## What is an AI Identity Provider? -Your [AI Models](/ai-gateway/entities/ai-model/) often need access control: some teams should reach certain AI Models and others should not, and you need a way to verify who is calling before a request consumes tokens or touches sensitive data. An AI Identity Provider lets you declare an inbound authentication mechanism at the gateway level and attach it to specific AI Models. +Your [AI Models](/ai-gateway/entities/ai-model/) and [AI Agents](/ai-gateway/entities/ai-agent/) often need access control: some teams should reach certain AI Models or AI Agents and others should not, and you need a way to verify who is calling before a request consumes tokens or touches sensitive data. An AI Identity Provider lets you declare an inbound authentication mechanism at the gateway level and attach it to specific AI Models or AI Agents. Use AI Identity Providers to: * Authenticate API keys and map them to [AI Consumers](/ai-gateway/entities/ai-consumer/) * Authenticate enterprise users through an existing identity provider (Okta, Azure AD, Google, or any OIDC-compliant IdP) without managing keys manually -* Apply different authentication to different models. For example, API keys for internal automation and OIDC bearer tokens for user-facing applications. +* Apply different authentication to different models or agents. For example, API keys for internal automation and OIDC bearer tokens for user-facing applications. -An AI Identity Provider manages inbound authentication, which is distinct from the outbound credentials managed by an [AI Model Provider](/ai-gateway/entities/ai-model-provider/). When an AI Consumer calls an AI Model, the AI Identity Provider checks who they are. The AI Model then uses the AI Model Provider's credentials to forward the request upstream. +An AI Identity Provider manages inbound authentication, which is distinct from the outbound credentials managed by an [AI Model Provider](/ai-gateway/entities/ai-model-provider/). When an AI Consumer calls an AI Model or AI Agent, the AI Identity Provider checks who they are. The AI Model then uses the AI Model Provider's credentials to forward the request upstream; an AI Agent proxies the now-authenticated request directly to its upstream agent. The following diagram shows where authentication fits in the request pipeline: @@ -178,12 +187,14 @@ The default `config.auth_methods` are `bearer` and `client_credentials`. If your To map the token to an existing AI Consumer, set `config.consumer_claims` to an array of path segments locating the claim in the token that carries the AI Consumer identifier (for example, `[["user", "info", "id"]]` to map to a nested `user.info.id` claim). If no mapping is needed, set `config.consumer_optional: true` to allow unauthenticated token holders through ACL checks. +`config.cache_tokens_salt` is required for `openid-connect` AI Identity Providers. It's a string used to generate the cache key for token endpoint request caching; set it to any unique value for this provider instance. + {:.warning} > All AI Models in the same {{site.ai_gateway}} that use OIDC authentication must reference the same `openid-connect` AI Identity Provider. Using different OIDC providers across models in the same {{site.ai_gateway}} is not supported. -## Assigning an AI Identity Provider to an AI Model +## Assigning an AI Identity Provider -An AI Identity Provider takes effect only when assigned to an [AI Model](/ai-gateway/entities/ai-model/). Reference the provider by `name` or `id` in the `access.identity_providers` array on the AI Model: +An AI Identity Provider takes effect only when assigned to an [AI Model](/ai-gateway/entities/ai-model/) or [AI Agent](/ai-gateway/entities/ai-agent/). Reference the provider by `name` or `id` in the entity's `access.identity_providers` array: ```yaml access: @@ -196,8 +207,9 @@ access: {:.info} > **Assignment rules** -> * Each AI Model supports one `key-auth` identity provider and one `openid-connect` identity provider. -> * You can assign both types to the same AI Model. A request is authenticated if it satisfies either provider. +> * Each AI Model supports one `key-auth` identity provider and one `openid-connect` identity provider. You can assign both types to the same AI Model; a request is authenticated if it satisfies either provider. +> * Each AI Agent currently supports up to one AI Identity Provider reference. +> * Attaching an authentication AI Policy directly to an AI Agent's `policies` field isn't supported. AI Identity Providers are the only supported way to authenticate AI Agent traffic. If you plan to rename the AI Identity Provider later, reference it by `id` rather than name. The ID is stable across renames. @@ -241,6 +253,7 @@ data: - bearer scopes: - openid + cache_tokens_salt: okta-ai-se-cache-salt {% endentity_example %} ## Schema diff --git a/app/_ai_gateway_entities/ai-model.md b/app/_ai_gateway_entities/ai-model.md index 8146f35af8e..96e1e1e47ff 100644 --- a/app/_ai_gateway_entities/ai-model.md +++ b/app/_ai_gateway_entities/ai-model.md @@ -69,7 +69,7 @@ faqs: - q: Can a client override the model name from the request body? a: | By default, no. The request `model` field must match the upstream model on one of the AI Model's targets, otherwise the runtime returns a `400` error. - To accept a client-side alias, set [`config.model.alias`](/ai-gateway/entities/ai-model/#schema-aigateway-model-config-model-alias). Clients can then send the alias value in the request `model` field instead of the upstream AI Provider model name. See [Request routing by model alias](/ai-gateway/load-balancing/#request-routing-by-model-alias) for details and examples. + To accept a client-side alias, configure [`config.route.model`](/ai-gateway/entities/ai-model/#schema-aigateway-model-config-route-model) with a routing rule. Clients can then send the alias value in the request body, a header, or the path instead of the upstream AI Provider model name. See [Request routing rules](#request-routing-rules) for details and examples. - q: Can a client override `temperature`, `top_p`, or `top_k` from the request? a: | @@ -218,6 +218,9 @@ rows: - format: "`huggingface`" provider: "[Hugging Face](/ai-gateway/ai-providers/huggingface/#supported-native-llm-formats-for-hugging-face)" capabilities: Text generation, streaming. + - format: "`vertex`" + provider: "[Vertex AI](/ai-gateway/ai-providers/vertex/)" + capabilities: Native Vertex AI request and response format. {% endtable %} @@ -309,7 +312,7 @@ For deeper background on vector storage and similarity matching, see [Embedding- Configure an embedding model to enable semantic routing. This lets {{site.ai_gateway}} route requests based on meaning and content similarity rather than just cost or latency. For example, route domain-specific queries to specialized providers or keep similar requests on the same provider for consistency. -Set [`config.balancer.embeddings`](#schema-aigateway-model-config-balancer-embeddings) to reference an AI Model Provider and embedding model name. Supported provider types: `azure`, `bedrock`, `databricks`, `gemini`, `huggingface`, `vercel`, `vertex`. The embedding model also powers the `semantic` load balancing algorithm. +Set [`config.balancer.embeddings`](#schema-aigateway-model-config-balancer-embeddings) to reference an AI Model Provider and embedding model name. Supported provider types: `azure`, `bedrock`, `gemini`, `huggingface`, `mistral`, `ollama`, `openai`, `vertex`. The embedding model also powers the `semantic` load balancing algorithm. ## Templating @@ -323,11 +326,19 @@ Substitution applies to the [`name`](#schema-aigateway-model-targets-name) of ea For examples of using templating, consult the {{site.ai_gateway}} documentation and API reference. -## Model aliasing +## Request routing rules -By default, applications or services making requests to the AI Model endpoint must specify the actual upstream model name (like `gpt-4o`) in the `model` field. If you want to allow them to use a different name, for abstraction, stability, or to hide implementation details, set [`config.model.alias`](#schema-aigateway-model-config-model-alias). +By default, applications or services making requests to the AI Model endpoint must specify the actual upstream model name (like `gpt-4o`) in the `model` field. If you want to allow them to use a different name, for abstraction, stability, or to hide implementation details, configure [`config.route.model`](#schema-aigateway-model-config-route-model) with a routing rule. -When an alias is set, clients can send that alias in the request `model` field instead of the upstream model name. This is useful when you want to decouple your client API from upstream provider changes. For example, you could expose an alias like `production-chat-model` while swapping the underlying upstream model from `gpt-4o` to `claude-3-sonnet` without your clients noticing. +`config.route.model` accepts one of the following match strategies: + +* [`body_param`](#schema-aigateway-model-config-route-model): matches a value in the request body, indexed by property name (for example, `body_param: model`). +* [`header_param`](#schema-aigateway-model-config-route-model): matches a value in the request headers, indexed by header name. +* [`path_param`](#schema-aigateway-model-config-route-model): matches a value present in the request path, indexed by path parameter name. + +Each strategy pairs with a [`values`](#schema-aigateway-model-config-route-model) array (currently limited to one value) listing the alias that routes to this AI Model. When a routing rule is set, clients can send the alias value (in the body, a header, or the path, depending on the strategy chosen) instead of the upstream model name. This is useful when you want to decouple your client API from upstream provider changes. For example, you could expose an alias like `production-chat-model` while swapping the underlying upstream model from `gpt-4o` to `claude-3-sonnet` without your clients noticing. + +When [`config.route.model`](#schema-aigateway-model-config-route-model) isn't set, {{site.ai_gateway}} creates a default model selector using the AI Model's name and format. ## Access control @@ -353,7 +364,7 @@ Use the [`config.proxy`](#schema-aigateway-model-config-proxy) object to specify ## Logging and observability -Enable [`statistics`](#schema-aigateway-model-config-logging-statistics) logging to track token consumption, request latency, and per-provider costs. This data flows into {{site.konnect_short_name}} analytics and any attached logging AI Policies, letting you monitor API spend, identify slow providers, and audit which AI Models drive the most usage. +{{site.ai_gateway}} automatically records token consumption, request latency, and per-provider costs for every AI Model, with no separate toggle required. This data flows into {{site.konnect_short_name}} analytics and any attached logging AI Policies, letting you monitor API spend, identify slow providers, and audit which AI Models drive the most usage. Optionally enable [`payloads`](#schema-aigateway-model-config-logging-payloads) to capture full request and response bodies. This is useful for debugging model responses, auditing sensitive operations, or replaying requests. @@ -393,15 +404,16 @@ data: route: paths: - /v1 + model: + body_param: model + values: + - my-gpt-4o logging: - statistics: true payloads: false - model: - alias: my-gpt-4o {% endentity_example %} {:.info} -> Because [`config.model.alias`](#schema-aigateway-model-config-model-alias) is set here, requests through this AI Model must send `"model": "my-gpt-4o"` (the alias) in the request body instead of the upstream target name (`gpt-4o`). Sending the upstream target name instead of the alias fails. +> Because [`config.route.model`](#schema-aigateway-model-config-route-model) is set here with `body_param: model`, requests through this AI Model must send `"model": "my-gpt-4o"` (the alias) in the request body instead of the upstream target name (`gpt-4o`). Sending the upstream target name instead of the alias fails. ## Schema diff --git a/app/_how-tos/ai-gateway/get-started-with-ai-gateway.md b/app/_how-tos/ai-gateway/get-started-with-ai-gateway.md index f679f8e955e..fc30f312b18 100644 --- a/app/_how-tos/ai-gateway/get-started-with-ai-gateway.md +++ b/app/_how-tos/ai-gateway/get-started-with-ai-gateway.md @@ -97,8 +97,10 @@ ai_gateway_models: route: paths: - /v1 - model: - alias: my-gpt-4o + model: + body_param: model + values: + - my-gpt-4o targets: - name: gpt-4o provider: generic-openai @@ -119,7 +121,7 @@ In this example, we're setting up the AI Model with: * `formats: [type: openai]`: Declares that this model accepts requests in OpenAI-compatible format. * `config.route.paths: [/v1]`: Configures the custom base path where this model's Routes will be accessible. Clients will send requests to paths that combine this base path with capability-specific Routes. * `capabilities: [generate]`: Enables the text generation capability. The `generate` capability creates a `/chat/completions` endpoint, so combined with your base path, clients send chat requests to `/v1/chat/completions`. -* `config.model.alias: my-gpt-4o`: Lets clients send `my-gpt-4o` in the request `model` field instead of the upstream model name. +* `config.route.model: { body_param: model, values: [my-gpt-4o] }`: Lets clients send `my-gpt-4o` in the request `model` field instead of the upstream model name. * `targets`: Specifies which upstream AI Provider model to route requests to. Here, `provider: generic-openai` references the AI Provider we created earlier, and `name: gpt-4o` specifies which OpenAI model to call upstream. ## Validate diff --git a/app/_how-tos/ai-gateway/use-claude-code-with-ai-gateway-anthropic.md b/app/_how-tos/ai-gateway/use-claude-code-with-ai-gateway-anthropic.md index b226976b490..83efde45ee8 100644 --- a/app/_how-tos/ai-gateway/use-claude-code-with-ai-gateway-anthropic.md +++ b/app/_how-tos/ai-gateway/use-claude-code-with-ai-gateway-anthropic.md @@ -74,8 +74,10 @@ ai_gateway_models: route: paths: - / - model: - alias: my-claude + model: + body_param: model + values: + - my-claude targets: - name: claude-opus-4-8 provider: generic-anthropic diff --git a/app/_how-tos/ai-gateway/use-claude-code-with-ai-gateway-azure.md b/app/_how-tos/ai-gateway/use-claude-code-with-ai-gateway-azure.md index 193123ab323..1d0838f4232 100644 --- a/app/_how-tos/ai-gateway/use-claude-code-with-ai-gateway-azure.md +++ b/app/_how-tos/ai-gateway/use-claude-code-with-ai-gateway-azure.md @@ -128,7 +128,7 @@ The AI Model uses: * `name`/`display_name: claude-code-azure-sonnet`: The identifier you pass to `claude --model`. {{ site.claude_code }} uses this, not the upstream target name, to select the model. * `formats: [type: anthropic]`: Declares that this model accepts requests in Anthropic-compatible format, matching what {{ site.claude_code }} sends natively. * `config.route.paths: [/]`: Configures the base path where this model's routes are accessible. -* `config.model.name_header: true`: Lets {{ site.claude_code }} select this model by sending its `name` in the request, instead of requiring a separate `alias`. +* `config.model.name_header: true`: Lets {{ site.claude_code }} select this model by sending its `name` in the request, instead of requiring a separate routing rule. * `capabilities: [generate]`: Enables text generation. For a model using the `anthropic` format, `generate` creates a `/messages` endpoint matching Anthropic's native Messages API, so combined with your base path, clients send requests to `/v1/messages`. * `policies`: Attaches the `claude-code-compat` policy created in the previous step, so its header and body transformations apply to every request sent through this model. * `targets`: Specifies which upstream model to route requests to. `provider: azure-claude` references the AI Provider created earlier, and `name: claude-sonnet-4-6` must match the name of your Claude deployment in Azure AI Foundry. diff --git a/app/_how-tos/ai-gateway/use-claude-code-with-ai-gateway-bedrock.md b/app/_how-tos/ai-gateway/use-claude-code-with-ai-gateway-bedrock.md index ed2c0ec2f9f..9c95a21e463 100644 --- a/app/_how-tos/ai-gateway/use-claude-code-with-ai-gateway-bedrock.md +++ b/app/_how-tos/ai-gateway/use-claude-code-with-ai-gateway-bedrock.md @@ -151,8 +151,10 @@ ai_gateway_models: route: paths: - / - model: - alias: my-claude-bedrock + model: + body_param: model + values: + - my-claude-bedrock targets: - name: us.anthropic.claude-haiku-4-5-20251001-v1:0 provider: my-aws-account diff --git a/app/_how-tos/ai-gateway/use-claude-code-with-ai-gateway-dashscope.md b/app/_how-tos/ai-gateway/use-claude-code-with-ai-gateway-dashscope.md index bee8e1dee96..0f7d96cfba4 100644 --- a/app/_how-tos/ai-gateway/use-claude-code-with-ai-gateway-dashscope.md +++ b/app/_how-tos/ai-gateway/use-claude-code-with-ai-gateway-dashscope.md @@ -75,8 +75,8 @@ ai_gateway_models: enabled: true formats: [{ type: anthropic }] config: - route: { paths: [/], methods: [GET, POST] } - model: { alias: qwen-plus, name_header: true } + route: { paths: [/], methods: [GET, POST], model: { body_param: model, values: [qwen-plus] } } + model: { name_header: true } capabilities: [generate] targets: - name: qwen-plus @@ -93,7 +93,7 @@ In this example we set: * `type: dashscope`: Connects to the Alibaba Cloud DashScope API as an AI Model Provider. * `capabilities: [generate]`: For a model using the `anthropic` format, `generate` creates a `/v1/messages` endpoint matching Anthropic's native Messages API. * `formats: [{ type: anthropic }]`: Accepts Anthropic-format requests to the AI Model entity, matching what {{ site.claude_code }} sends. - * `config.model.alias: qwen-plus`: The model name {{ site.claude_code }} should send in each request, which you can set with the `ANTHROPIC_MODEL` variable or `--model` option. + * `config.route.model: { body_param: model, values: [qwen-plus] }`: The model name {{ site.claude_code }} should send in each request, which you can set with the `ANTHROPIC_MODEL` variable or `--model` option. * `route.paths: [/]`: Configures the custom base path where this model's routes will be accessible. Setting this to a unique value avoids clashes when you have multiple AI Models. * `targets[0].config.international: true`: Uses DashScope's international endpoint (`dashscope-intl.aliyuncs.com`). This is the default. If your DashScope key belongs to a mainland China account, set this to `false` so requests reach `dashscope.aliyuncs.com` instead. diff --git a/app/_how-tos/ai-gateway/use-claude-code-with-ai-gateway-gemini.md b/app/_how-tos/ai-gateway/use-claude-code-with-ai-gateway-gemini.md index 4f1560b84e3..92c8a72632d 100644 --- a/app/_how-tos/ai-gateway/use-claude-code-with-ai-gateway-gemini.md +++ b/app/_how-tos/ai-gateway/use-claude-code-with-ai-gateway-gemini.md @@ -151,8 +151,10 @@ ai_gateway_models: route: paths: - / - model: - alias: my-claude-gemini + model: + body_param: model + values: + - my-claude-gemini targets: - name: gemini-2.5-flash provider: !ref my-gemini-account#name diff --git a/app/_how-tos/ai-gateway/use-claude-code-with-ai-gateway-huggingface.md b/app/_how-tos/ai-gateway/use-claude-code-with-ai-gateway-huggingface.md index 4b507f33150..9b10c4ef151 100644 --- a/app/_how-tos/ai-gateway/use-claude-code-with-ai-gateway-huggingface.md +++ b/app/_how-tos/ai-gateway/use-claude-code-with-ai-gateway-huggingface.md @@ -136,8 +136,10 @@ ai_gateway_models: route: paths: - / - model: - alias: my-huggingface + model: + body_param: model + values: + - my-huggingface targets: - name: meta-llama/Llama-3.3-70B-Instruct provider: !ref my-huggingface-account#name diff --git a/app/_how-tos/ai-gateway/use-claude-code-with-ai-gateway-openai.md b/app/_how-tos/ai-gateway/use-claude-code-with-ai-gateway-openai.md index e5f39c01299..fc6b9d91266 100644 --- a/app/_how-tos/ai-gateway/use-claude-code-with-ai-gateway-openai.md +++ b/app/_how-tos/ai-gateway/use-claude-code-with-ai-gateway-openai.md @@ -80,8 +80,10 @@ ai_gateway_models: route: paths: - / - model: - alias: my-claude-openai + model: + body_param: model + values: + - my-claude-openai targets: - name: gpt-5-mini provider: generic-openai diff --git a/app/_how-tos/ai-gateway/use-claude-code-with-ai-gateway-vertex.md b/app/_how-tos/ai-gateway/use-claude-code-with-ai-gateway-vertex.md index 720e3ffa7cb..4f21c9bc274 100644 --- a/app/_how-tos/ai-gateway/use-claude-code-with-ai-gateway-vertex.md +++ b/app/_how-tos/ai-gateway/use-claude-code-with-ai-gateway-vertex.md @@ -143,7 +143,7 @@ The AI Model uses: * `name`/`display_name: claude-code-vertex-sonnet`: The identifier you pass to `claude --model`. {{ site.claude_code }} uses this, not the upstream target ID, to select the model. * `formats: [type: anthropic]`: Accepts Anthropic-compatible requests (what {{ site.claude_code }} sends). - * `config.model.name_header: true`: Lets {{ site.claude_code }} select this model by sending its `name` in the request, instead of requiring a separate `alias`. + * `config.model.name_header: true`: Lets {{ site.claude_code }} select this model by sending its `name` in the request, instead of requiring a separate routing rule. * `capabilities: [generate]`: Enables text generation. For a model using the `anthropic` format, `generate` creates a `/messages` endpoint matching Anthropic's native Messages API. * `policies`: Attaches the `claude-code-compat` policy defined above, via `!ref claude-code-compat#name`, so its body-stripping transformation applies to every request sent through this model. * `targets[0].provider: vertex-prod`: Routes upstream requests through the Vertex AI Provider created earlier. diff --git a/app/_how-tos/ai-gateway/use-codex-with-ai-gateway.md b/app/_how-tos/ai-gateway/use-codex-with-ai-gateway.md index 04ad8cb2f62..e1107319b19 100644 --- a/app/_how-tos/ai-gateway/use-codex-with-ai-gateway.md +++ b/app/_how-tos/ai-gateway/use-codex-with-ai-gateway.md @@ -76,8 +76,8 @@ ai_gateway_models: enabled: true formats: [{ type: openai }] config: - route: { paths: [/codex], methods: [GET, POST] } - model: { alias: gpt-5.4, name_header: true } + route: { paths: [/codex], methods: [GET, POST], model: { body_param: model, values: [gpt-5.4] } } + model: { name_header: true } capabilities: [agentic] targets: - name: gpt-5.4 @@ -92,7 +92,7 @@ In this example: * `type: openai`: Connects to the OpenAI API. * `capabilities: [agentic]`: Routes requests to the OpenAI Responses API, which the Codex CLI uses. * `formats: [{ type: openai }]`: Accepts OpenAI-format requests. - * `config.model.alias: gpt-5.4`: The model name the Codex CLI sends in each request. + * `config.route.model: { body_param: model, values: [gpt-5.4] }`: The model name the Codex CLI sends in each request. * `route.paths: [/codex]`: The base path Codex points at; the Responses API is served at `/codex/responses`. ## Verify the AI Model diff --git a/app/ai-gateway/configure-on-prem.md b/app/ai-gateway/configure-on-prem.md index 0099bad1041..1e928e32a04 100644 --- a/app/ai-gateway/configure-on-prem.md +++ b/app/ai-gateway/configure-on-prem.md @@ -146,7 +146,8 @@ The following examples walk through converting a decK `ai.yaml` file for a singl paths: - /ai model: - path_aliases: ["@openai/gpt-5.2"] + body_param: model + values: ["@openai/gpt-5.2"] targets: - name: gpt-5.2 provider: openai-prod @@ -225,6 +226,9 @@ The following examples walk through converting a decK `ai.yaml` file for a singl {: .no-copy-code .collapsible } The AI Provider generates no object of its own. Its `type` becomes the target's `model.provider`, and its `auth` is materialized into the same `ai-proxy-advanced` target. + + {:.info} + > The converted output uses the `alias` and `model_alias` field names from the self-hosted `ai-model-selector` and `ai-proxy-advanced` plugin schemas. These are distinct from `config.route.model` on the {{site.ai_gateway}} entity shown in the input above; `deck file ai2kong` handles the translation between the two. 1. Sync the converted config to your self-hosted {{site.base_gateway}}: ```sh deck gateway sync kong.yaml @@ -391,7 +395,8 @@ An AI Policy generates no object of its own. It must be attached to another enti paths: - /ai model: - path_aliases: ["@openai/gpt-5.2"] + body_param: model + values: ["@openai/gpt-5.2"] policies: - ai-gw-prompt-guard targets: diff --git a/app/ai-gateway/load-balancing.md b/app/ai-gateway/load-balancing.md index e45f4c3e752..0477f456bfb 100644 --- a/app/ai-gateway/load-balancing.md +++ b/app/ai-gateway/load-balancing.md @@ -107,11 +107,11 @@ rows: For examples of each algorithm, see [Algorithm examples](/ai-gateway/entities/ai-model/#algorithm-examples) in the [AI Model entity](/ai-gateway/entities/ai-model/) reference. -### Request routing by model alias +### Request routing rules -Model aliases allow clients to send an alias instead of the actual model name in the request. This decouples the external model identifier from the internal provider model, enabling flexible routing without changing client code. +Routing rules allow clients to send an alias instead of the actual model name in the request. This decouples the external model identifier from the internal provider model, enabling flexible routing without changing client code. -Each target in a Model entity can have an optional [`model.alias`](/ai-gateway/entities/ai-model/#schema-aigateway-model-target-models-model-alias) field. When a client sends `"model": "alias-value"` in the request body, {{site.ai_gateway}} routes to the matching target. This feature works independently of load balancing algorithms — the alias determines which target (or set of targets) handles the request, and the configured load balancing algorithm selects the final backend within that set. +An AI Model can have an optional [`config.route.model`](/ai-gateway/entities/ai-model/#schema-aigateway-model-config-route-model) field, matching on the request body, a header, or the path. For example, when `config.route.model` is set to `{ body_param: model, values: [alias-value] }` and a client sends `"model": "alias-value"` in the request body, {{site.ai_gateway}} routes to the matching AI Model. This feature works independently of load balancing algorithms. The routing rule determines which AI Model (or target set) handles the request, and the configured load balancing algorithm selects the final backend within that set. ### Retry and fallback