Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
6 changes: 3 additions & 3 deletions src/lib/content/concepts/rollouts-and-traffic/ingress.md
Original file line number Diff line number Diff line change
Expand Up @@ -98,7 +98,7 @@ For `llama-chat`, OME creates up to four HTTPRoutes, all with the hostname `llm.
| `llama-chat-router` | `/llama-demo/llama-chat-router/` | `llama-chat-router`, with a router |
| `llama-chat-decoder` | `/llama-demo/llama-chat-decoder/` | `llama-chat-decoder`, with a decoder |

Clients use the top-level route, `llama-chat`. The other routes each reach one component, bypassing the router. OME creates a component's route once the component is ready.
Clients use the top-level route, `llama-chat`. The other routes each reach one component, bypassing the router. OME creates a component's route once the component is ready. For an OMENative component, a [serving Instance](../omenative/instances.md#readiness-and-availability) is enough: OME creates and updates the route even while the component's `EngineReady`, `RouterReady` or `DecoderReady` condition is `False`, as long as the component's [lifecycle](../architecture/deployment-modes.md#lifecycle-status) reports `readyReplicas` and `servingReplicas` above `0`. So a component whose [condition](../architecture/deployment-modes.md#conditions) turns `False` with the reason `InsufficientAvailable` keeps its route, and `IngressReady` can stay `True`, while OME restores the Instances.

Each route rewrites its path prefix to `/`, so a request for `/llama-demo/llama-chat/v1/models` reaches the model server as `/v1/models`. It also sets the headers `OMe-Isvc-Name` and `OME-Isvc-Namespace` to the InferenceService's name and namespace.

Expand Down Expand Up @@ -199,13 +199,13 @@ The InferenceService is `Ready` only when `IngressReady` is `True`, as [Status](
| --- | --- |
| The external Service | Always, with the reason `IngressDisabled`. |
| A Kubernetes Ingress | OME has created the Ingress. It waits for the router, or the decoder when there's no router, or else the engine. It doesn't check your ingress controller. |
| HTTPRoutes | Every declared component is ready. The Gateway has accepted every HTTPRoute, with no `False` condition. The Gateway can reject the default BackendTrafficPolicy without changing `IngressReady`. |
| HTTPRoutes | Every declared component is ready, or is an OMENative component with a serving Instance. The Gateway has accepted every HTTPRoute, with no `False` condition. The Gateway can reject the default BackendTrafficPolicy without changing `IngressReady`. |

When `IngressReady` is `False`, its reason and message say why:

| Reason | Message | Meaning |
| --- | --- | --- |
| `ComponentNotReady` | `Target service not ready for ingress creation`, or `engine component not ready for HTTPRoute creation` | OME creates the Ingress or the HTTPRoute once the component is ready. |
| `ComponentNotReady` | `Target service not ready for ingress creation`, or `engine component not ready for HTTPRoute creation` | OME creates the Ingress once the component is ready, and the HTTPRoute once the component is ready or, for an OMENative component, has a serving Instance: see [Routes](#routes). |
| `ParentStatusNotAvailable` | `engine HTTPRoute awaiting gateway programming`, or `Engine HttpRouteNotReady` | The Gateway hasn't reported on the HTTPRoute yet. |
| The Gateway's reason, such as `NotAllowedByListeners` | `Engine`, then the Gateway's message | The Gateway reports a `False` condition on the HTTPRoute. |

Expand Down
2 changes: 1 addition & 1 deletion src/lib/content/guides/networking/configure-ingress.md
Original file line number Diff line number Diff line change
Expand Up @@ -367,7 +367,7 @@ True IngressDisabled Ingress creation is disabled, using external service f
### OME doesn't create a route

- With the reason `IngressDisabled`, ingress creation is off. Check the ConfigMap as in Step 1, and look for the `ome.io/ingress-disable-creation` annotation on the InferenceService. If the ConfigMap has `"disableIngressCreation": false`, restart the controller.
- With the reason `ComponentNotReady`, the router, decoder or engine isn't ready, and OME creates its route when it is. See [Troubleshoot an InferenceService](../troubleshoot/troubleshoot-an-inferenceservice.md).
- With the reason `ComponentNotReady`, the router, decoder or engine isn't ready, and OME creates its route when it is. With Gateway API, an OMENative component needs only a serving Instance for its route, so the reason means it has none, as [Routes](../../concepts/rollouts-and-traffic/ingress.md#routes) describes. See [Troubleshoot an InferenceService](../troubleshoot/troubleshoot-an-inferenceservice.md).
- If the condition is missing, or doesn't change after you restart the controller, an error stops OME before it updates the status. Read the logs of every controller replica:

```bash
Expand Down
2 changes: 1 addition & 1 deletion src/lib/content/guides/networking/multiple-gateways.md
Original file line number Diff line number Diff line change
Expand Up @@ -86,7 +86,7 @@ llama-chat internal-gateway,external-gateway llm.internal.example.com
llama-chat-engine internal-gateway,external-gateway llm.internal.example.com,llm.external.example.com
```

The routes list the primary gateway first, with one hostname per gateway in the same order. A router or a decoder gets its own route too, on the same gateways. OME creates a component's route only once the component is ready.
The routes list the primary gateway first, with one hostname per gateway in the same order. A router or a decoder gets its own route too, on the same gateways. OME creates a component's route once the component is ready, or, for an [OMENative](../../concepts/omenative/overview.md) component, once it has a serving Instance: see [Routes](../../concepts/rollouts-and-traffic/ingress.md#routes).

The listener hostnames from [Before you begin](#before-you-begin) keep each gateway to its own hostname: a listener with a hostname [ignores the route's hostnames that don't match it](https://gateway-api.sigs.k8s.io/api-types/httproute/#hostnames). A listener without one would serve both.

Expand Down