From 7ad97abad7a1d53187e331fc2f7e2b60207aec8f Mon Sep 17 00:00:00 2001 From: bong-water-water-bong Date: Fri, 13 Nov 2026 05:02:07 -0400 Subject: [PATCH] docs/npu: /api/v1/health is a 404, and curl -s exits 0 on a 404, so that probe always "passes" Found by running the NPU fast lane on Qwen3-0.6B and writing the readiness probe from Lemonade's API shape: curl -s http://127.0.0.1:18431/api/v1/health -> 404, exit 0, script says "ready" curl -sf http://127.0.0.1:18432/health -> 503 while loading, then 200 So the readiness check passed while the weights were still loading, and the first chat request came back {"error":{"message":"model is loading"}} - which I then compensated for with retries, hiding the real bug. With `-f` and the right path the same smoke answers on the **first** request. The page already said `/v1/health` is "503 while loading"; it did not say there is no `/api/v1/*` prefix, which is what a lane copying the other API gets. `scripts/npu-window11.sh:430` has the right form (`curl -sf ... /health`) - this sentence is the one-line version of why that form is right. Additive only: 4 lines +1 reflowed, no headings touched, `doc_section_loss_check` -> all 644 section(s) present, copyright --check rc 0. --- docs/npu.md | 6 +++++- 1 file changed, 5 insertions(+), 1 deletion(-) diff --git a/docs/npu.md b/docs/npu.md index 64c1b8dc0..2b2c6f165 100644 --- a/docs/npu.md +++ b/docs/npu.md @@ -2681,7 +2681,11 @@ that runs the phase is private, like the kernels it runs. `tests/data/qwen3_tokenizer_golden.json` holds 14 cases encoded by `tokenizers` 0.23.1, and all 14 match, ids and round trip. - **Endpoints:** `/v1/health` (503 while loading), `/v1/models`, `/v1/chat/completions` - (streaming or not) and `/v1/completions`. + (streaming or not) and `/v1/completions`. The serve also answers `/health` beside `/v1/health`, and there is + **no `/api/v1/*` prefix**: a probe against `/api/v1/health` is a 404, `curl -s` exits 0 on a 404, and a + readiness check written that way therefore passes while the weights are still loading -- the request then + comes back `{"error":{"message":"model is loading"}}`. Use `curl -sf /health` (measured: `/api/v1/health` + 404, `/health` 503 while loading then 200; `scripts/npu-window11.sh` is the house form of this probe). - **Prompt:** the **container's own chat template** — `tokenizer_config.json`'s `chat_template` string, else `chat_template.jinja` beside it — rendered with the pinned llama.cpp fork's Jinja engine (`app/npu_template.h`, `docs/npu-chat-template.md`). A container that declares no template keeps ChatML