Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions docs/en/changes/changes.md
Original file line number Diff line number Diff line change
Expand Up @@ -29,6 +29,7 @@

#### Documentation
* Document the BanyanDB trace tail sampling metrics in the BanyanDB self-observability dashboard catalog, and point the "Operating it" section of the trace tail sampling guide at them — the OAP-collected metrics show what the sampler plugins *proposed* next to what storage *committed*, which the data node's raw metrics endpoint alone does not.
* Rename Envoy AI Gateway to Agent Router in the monitoring docs, following the project's move to the Agentic AI Foundation. Only the prose changes: the `ENVOY_AI_GATEWAY` layer, the `job_name=envoy-ai-gateway` routing tag, the `envoy-ai-gateway` rule files and the `meter_envoy_ai_gw_` metric names are kept, as Agent Router itself kept its deployed names and the telemetry it emits.


All issues and pull requests are [here](https://github.com/apache/skywalking/issues?q=milestone:11.1.0)
Expand Down
40 changes: 26 additions & 14 deletions docs/en/setup/backend/backend-envoy-ai-gateway-monitoring.md
Original file line number Diff line number Diff line change
@@ -1,23 +1,35 @@
# Envoy AI Gateway Monitoring
# Agent Router (formerly Envoy AI Gateway) Monitoring

## Envoy AI Gateway observability via OTLP
## Agent Router observability via OTLP

[Envoy AI Gateway](https://aigateway.envoyproxy.io/) is a gateway/proxy for AI/LLM API traffic
(OpenAI, Anthropic, AWS Bedrock, Azure OpenAI, Google Gemini, etc.) built on top of Envoy Proxy.
[Agent Router](https://theagentrouter.ai/), formerly Envoy AI Gateway, is a gateway/proxy for AI/LLM API traffic
(OpenAI, Anthropic, AWS Bedrock, Azure OpenAI, Google Gemini, etc.) built on top of Envoy Proxy and Envoy Gateway.
It natively emits GenAI metrics following
[OpenTelemetry GenAI Semantic Conventions](https://opentelemetry.io/docs/specs/semconv/gen-ai/),
and also emits MCP (Model Context Protocol) metrics and access logs via OTLP.

SkyWalking receives OTLP metrics and logs directly on its gRPC port (11800) — no OpenTelemetry
Collector is needed between the AI Gateway and SkyWalking OAP.
Collector is needed between Agent Router and SkyWalking OAP.

### Names kept from Envoy AI Gateway

This integration was built while the project was named Envoy AI Gateway. The rename changed only the product name:
Agent Router kept the names you deploy (the `aigateway.envoyproxy.io` API group, the `aigw` CLI) and emits the same
telemetry. SkyWalking keeps its names unchanged in the same way, so existing gateway deployments, stored data,
dashboards and alarm rules keep working:

- the layer `ENVOY_AI_GATEWAY`
- the routing tag `job_name=envoy-ai-gateway`
- the rule files `otel-rules/envoy-ai-gateway/*` and `lal/envoy-ai-gateway.yaml`
- the metric names prefixed with `meter_envoy_ai_gw_`

### Prerequisites
- [Envoy AI Gateway](https://aigateway.envoyproxy.io/) deployed. See the
[Envoy AI Gateway getting started](https://aigateway.envoyproxy.io/docs/getting-started/) for installation.
- [Agent Router](https://theagentrouter.ai/) deployed. See the
[Agent Router getting started](https://theagentrouter.ai/docs/getting-started/) for installation.

### Data flow
1. Envoy AI Gateway processes LLM API requests and MCP requests, recording GenAI metrics and MCP metrics.
2. The AI Gateway pushes metrics and access logs via OTLP gRPC to SkyWalking OAP.
1. Agent Router processes LLM API requests and MCP requests, recording GenAI metrics and MCP metrics.
2. Agent Router pushes metrics and access logs via OTLP gRPC to SkyWalking OAP.
3. SkyWalking OAP parses metrics with [MAL](../../concepts-and-designs/mal.md) rules and access logs
with [LAL](../../concepts-and-designs/lal.md) rules.

Expand All @@ -26,7 +38,7 @@ Collector is needed between the AI Gateway and SkyWalking OAP.
The MAL rules (`envoy-ai-gateway/*`) and LAL rules (`envoy-ai-gateway`) are enabled by default
in SkyWalking OAP. No OAP-side configuration is needed.

Configure the AI Gateway to push OTLP to SkyWalking by setting these environment variables:
Configure Agent Router to push OTLP to SkyWalking by setting these environment variables:

| Env Var | Value | Purpose |
|-------------------------------|-----------------------------------------------------|------------------------------|
Expand All @@ -38,9 +50,9 @@ Configure the AI Gateway to push OTLP to SkyWalking by setting these environment
| `OTEL_RESOURCE_ATTRIBUTES` | See below | Routing + instance + layer |

**Required resource attributes** (in `OTEL_RESOURCE_ATTRIBUTES`):
- `job_name=envoy-ai-gateway` — Fixed routing tag for MAL/LAL rules. Same for all AI Gateway deployments.
- `job_name=envoy-ai-gateway` — Fixed routing tag for MAL/LAL rules. Same for all Agent Router deployments.
- `service.instance.id=<instance-id>` — Instance identity. In Kubernetes, use the pod name via Downward API.
- `service.layer=ENVOY_AI_GATEWAY` — Routes access logs to the AI Gateway LAL rules.
- `service.layer=ENVOY_AI_GATEWAY` — Routes access logs to the Agent Router LAL rules.

**Example:**
```bash
Expand All @@ -54,7 +66,7 @@ OTEL_RESOURCE_ATTRIBUTES=job_name=envoy-ai-gateway,service.instance.id=pod-abc12

### Supported Metrics

SkyWalking observes the AI Gateway as a `LAYER: ENVOY_AI_GATEWAY` service. Each gateway deployment
SkyWalking observes Agent Router as a `LAYER: ENVOY_AI_GATEWAY` service. Each gateway deployment
is a service, each pod is an instance. Metrics include per-provider and per-model breakdowns.

#### Service Metrics
Expand Down Expand Up @@ -96,7 +108,7 @@ including per-provider and per-model breakdowns.

### MCP Metrics

When the AI Gateway is configured with MCP (Model Context Protocol) routes, SkyWalking collects
When Agent Router is configured with MCP (Model Context Protocol) routes, SkyWalking collects
MCP-specific metrics. These appear in the **MCP** tab on the service and instance dashboards.

#### MCP Service Metrics
Expand Down
2 changes: 1 addition & 1 deletion docs/en/setup/backend/marketplace.md
Original file line number Diff line number Diff line change
Expand Up @@ -35,7 +35,7 @@ the matching dashboard / visualization lives in Horizon UI.
- **Infrastructure** - Linux and Windows server monitoring
- **Cloud Services** - AWS EKS, S3, DynamoDB, API Gateway, and more
- **Gateways** - Nginx, APISIX, Kong monitoring
- **GenAI** - [Virtual GenAI](../service-agent/virtual-genai.md) for agent-based LLM call monitoring, [Envoy AI Gateway](backend-envoy-ai-gateway-monitoring.md) for infrastructure-side AI traffic observability
- **GenAI** - [Virtual GenAI](../service-agent/virtual-genai.md) for agent-based LLM call monitoring, [Agent Router (formerly Envoy AI Gateway)](backend-envoy-ai-gateway-monitoring.md) for infrastructure-side AI traffic observability
- **Databases** - MySQL, PostgreSQL, Redis, Elasticsearch, MongoDB, ClickHouse, and more
- **Message Queues** - Kafka, RabbitMQ, Pulsar, RocketMQ, ActiveMQ
- **Browser** - Real user monitoring for web applications
Expand Down
2 changes: 1 addition & 1 deletion docs/en/setup/backend/opentelemetry-receiver.md
Original file line number Diff line number Diff line change
Expand Up @@ -41,7 +41,7 @@ in the resource attributes, the fallback is skipped.
| `net.host.name` | `node_identifier_host_name` | Legacy: used by VM/Windows MAL rules |
| `host.name` | `node_identifier_host_name` | Legacy: used by VM/Windows MAL rules |

When `job_name` is set explicitly in `OTEL_RESOURCE_ATTRIBUTES` (e.g., by Envoy AI Gateway),
When `job_name` is set explicitly in `OTEL_RESOURCE_ATTRIBUTES` (e.g., `job_name=envoy-ai-gateway` for [Agent Router](backend-envoy-ai-gateway-monitoring.md)),
it takes precedence and the `service.name` fallback is skipped.

**Note:** The `net.host.name` and `host.name` mappings are legacy. New integrations should use
Expand Down
2 changes: 1 addition & 1 deletion docs/menu.yml
Original file line number Diff line number Diff line change
Expand Up @@ -168,7 +168,7 @@ catalog:
path: "/en/setup/backend/ai-evaluation"
- name: "AI Agent Conversations"
path: "/en/setup/backend/ai-agent-conversation"
- name: "Envoy AI Gateway"
- name: "Agent Router (Envoy AI Gateway)"
path: "/en/setup/backend/backend-envoy-ai-gateway-monitoring"
- name: "Self Observability"
catalog:
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -284,8 +284,8 @@ public final class Layer {
public static final Layer VIRTUAL_GENAI = register("VIRTUAL_GENAI", 45, false);

/**
* Envoy AI Gateway is an AI/LLM traffic gateway built on Envoy Proxy,
* providing observability for GenAI API traffic.
* Agent Router, formerly Envoy AI Gateway, is an AI/LLM traffic gateway built on Envoy Proxy,
* providing observability for GenAI API traffic. The layer name predates the rename and is kept.
*/
public static final Layer ENVOY_AI_GATEWAY = register("ENVOY_AI_GATEWAY", 46, true);

Expand Down
Loading