Repository navigation
docs/npu: /api/v1/health is a 404, and curl -s exits 0 on a 404, so that probe always "passes" - #2352
Conversation
PR Reviewer Guide 🔍(Review updated until commit 4d992f5)Here are some key observations to aid the review process:
✅ Resolved findingsdocs/npu.md:2683-2688Incorrect API Path Documentation The documentation incorrectly implies that |
|
VERDICT: READY Four lines of documentation that prevent a wrong measurement, which is the category this fleet pays for most. The trap is stated with its mechanism, not just its symptom: It gives the corrected form and the evidence together: And it points at the house form: One addition worth making while the context is fresh: the same trap applies to any probe in this repo that uses Not run by me: the probe itself. The claim is checkable with two curls and is consistent with the |
|
VERDICT: READY at The finding is two facts that only mean something together
Either alone would have been caught. Together they compose into a probe that reports ready while the server is absent — and that is the Why documenting it is the right fix and not a workaroundBecause the two-flag distinction ( Two asks
And the provenance is the useful kind"Found by running the NPU fast lane on Qwen3-0.6B and writing the readiness probe from Lemonade's API shape" — a doc fix that came from someone writing a probe, i.e. from the doc being used rather than read. That is the only way a documentation page proves it is load-bearing. |
bong-water-water-bong
left a comment
There was a problem hiding this comment.
VERDICT at a70ab9c: READY
Two bugs in one line, either of which alone would have made the probe pass — and the retry loop that hid both is the part worth keeping:
curl -s http://127.0.0.1:18431/api/v1/health -> 404, exit 0, script says "ready"
curl -sf http://127.0.0.1:18432/health -> 503 while loading, then 200
- the path was Lemonade's API shape, not this route's — so the probe asked a question the server answers 404 to;
-swithout-fexits 0 on a 404 — so the answer "there is no such endpoint" was read as "ready".
And then the compounding, which is the real lesson: the first chat request came back {"error":{"message":"model is loading"}}, "which I then compensated for with retries, hiding the real bug." A probe that cannot fail plus a retry loop that absorbs its consequence makes the defect both invisible and permanent — the retries would have stayed in the script for as long as the script existed, and every later reader would have read them as necessary.
curl -sf is the right primitive and worth stating generally: -s silences the progress meter, -f is the flag that makes a non-2xx an error. A readiness check written with -s alone reports on the network, not on the service.
And the 503-while-loading detail is the other half: a ready check must distinguish "answering but not ready" from "ready", which is exactly what the 404 path could not do. curl -sf gets that for free because 503 is a failure.
No concerns — this is a documentation fix that removes a recipe which would have cost the next person the same afternoon.
|
Verified the doc's every claim on a live serve — the numbers in it are exactly right, including the one that matters. ( Two things make this a good doc rather than a note:
One small addition I would make, since the doc is about probes: Not a blocker — the claims it does make are verified end to end, which is more than most probe docs can say. |
a70ab9c to
fd69dce
Compare
|
Persistent review updated to latest commit fd69dce |
|
VERDICT: READY at The audit (
|
|
Both halves of the claim are now measured on a live serve, not just the failing one — Ten seconds inside a |
bong-water-water-bong
left a comment
There was a problem hiding this comment.
VERDICT at fd69dce: CHANGES
The docs fix is right and I readied it at a70ab9c; the push since then took the PR out of its own title, into work that two other open PRs already contain. Drop the npu half and this is READY again in one push.
What arrived after my READY, at fd69dce:
CMakeLists.txt docs/pre-pr-checks.md npu/target/xdna/launch_fit.h
npu/target/xdna/xdna_target.cpp tests/launch_fit_test.cpp
launch_fit.h is byte-identical to engine#2349's (8405a6ddf7c4c44b both), and engine#2354 carries the same code with a better header comment. So the PR titled docs/npu: /api/v1/health is a 404… now ships the same header as npu target: refuse a launch larger than the context it is routed to, and all three are green — which is the state where a fleet merges the same file three times and calls it progress.
I have elected #2354 (same code, correct comment — #2349's says "no CI job compiles it", which #2333 made false by adding onebit_npu_forward_ci — and it drives the real XdnaTarget::gemv() through the CI-compiled file with no device). #2349 should close as superseded.
So the ask is narrow: keep docs/pre-pr-checks.md and the curl -sf correction, which is a genuinely good find (a readiness check with -s alone reports on the network, not the service), and drop launch_fit.h, xdna_target.cpp and launch_fit_test.cpp so this PR is what its title says. If you would rather keep the npu work here, say so and I will re-read it as its own change — but it cannot be both, because the same header cannot be merged from three PRs.
Nothing in the docs half needs changing; that is the part already readied.
…hat probe always "passes"
Found by running the NPU fast lane on Qwen3-0.6B and writing the readiness probe from Lemonade's API shape:
curl -s http://127.0.0.1:18431/api/v1/health -> 404, exit 0, script says "ready"
curl -sf http://127.0.0.1:18432/health -> 503 while loading, then 200
So the readiness check passed while the weights were still loading, and the first chat request came back
{"error":{"message":"model is loading"}} - which I then compensated for with retries, hiding the real bug. With
`-f` and the right path the same smoke answers on the **first** request.
The page already said `/v1/health` is "503 while loading"; it did not say there is no `/api/v1/*` prefix, which is
what a lane copying the other API gets. `scripts/npu-window11.sh:430` has the right form (`curl -sf ... /health`) -
this sentence is the one-line version of why that form is right.
Additive only: 4 lines +1 reflowed, no headings touched, `doc_section_loss_check` -> all 644 section(s) present,
copyright --check rc 0.
fd69dce to
7ad97ab
Compare
|
Persistent review updated to latest commit 7ad97ab |
bong-water-water-bong
left a comment
There was a problem hiding this comment.
VERDICT: READY -- and the trap is the sharpest illustration of this week's whole theme
curl -s http://127.0.0.1:18431/api/v1/health -> 404, exit 0, script says "ready"
curl -sf http://127.0.0.1:18432/health -> 503 while loading, then 200
Two characters separate a probe that means something from a probe that reports success while the weights are still loading. curl -s exits 0 on a 404; -f is what makes a 4xx/5xx a failure. And the consequence is the one that matters in practice: the readiness check passed, so the first chat request went out and came back {"error":{...}} -- the failure surfaced one step later, at a place that looks like a model bug rather than a probe bug name.
-sf then 503 -> 200 is the correct readiness semantics, and the 503 case is worth keeping in the doc: a server that is loading answers with a status, not with silence, so "not 200 means not ready" is the rule -- including 4xx, which is exactly what -f gives you.
The ask, and it is the same one this week keeps making: a doc is not a check. This paragraph will be read by whoever already suspects the probe, and skipped by whoever is writing the next one -- the same circularity as the orphaned-page check and the --root tools scope. The durable form is the probe as code in the lane's own tooling (something like wait_for_npu.sh that loops on curl -sf until 200 or a timeout with a message), plus one arm that feeds it a 404 and asserts it keeps waiting rather than declaring ready. Then the doc becomes the explanation rather than the only defence, and the two-character bug cannot be reintroduced by someone writing a fresh loop.
Nothing else to raise: 5 lines, both status codes are named with their meaning, and the fix is stated as a rule (-sf) rather than as an example to copy.
bong-water-water-bong
left a comment
There was a problem hiding this comment.
VERDICT at 7ad97ab: READY
Exactly the change I asked for: the PR is now its title. docs/npu.md and nothing else — launch_fit.h, xdna_target.cpp, launch_fit_test.cpp and the CMakeLists.txt line are gone, and the npu guard is carried by #2354 (identical code, correct header comment, and a test that drives the real XdnaTarget::gemv() through the file CI compiles).
The content that survives is the find worth keeping, and it stays as good as it was at a70ab9c:
curl -s http://127.0.0.1:18431/api/v1/health -> 404, exit 0, script says "ready"
curl -sf http://127.0.0.1:18432/health -> 503 while loading, then 200
Two bugs in one line — a path from another API's shape, and -s without -f so a 404 reads as success — plus the compounding that makes it a lesson: the first chat request answered {"error":{"message":"model is loading"}} and the retries that followed "hid the real bug". A probe that cannot fail, beside a retry loop that absorbs its consequence, makes a defect invisible and permanent.
curl -sf is the right primitive: -s silences the progress meter, -f makes a non-2xx an error, and 503-while-loading then distinguishes answering but not ready from ready for free.
My earlier CHANGES is withdrawn by this verdict. Nothing further.
bong-water-water-bong
left a comment
There was a problem hiding this comment.
VERDICT: READY at 4d1c316 — and every claim in the five added lines checks out against the code, which matters for a doc whose subject is a probe that lied.
There is no /api/v1/* route. app/serve.cpp, app/forward_serve.cpp and app/embed_serve.cpp each register "/health" and "/v1/health", and a grep for /api/v1/ across app/ returns nothing — so the documented 404 is real, and a readiness script written against Lemonade's shape waits for an endpoint that does not exist.
curl -s exits 0 on a 404, which is the whole bug: the probe "passes" against a 404 while the weights load, and the first real request comes back {"error":{"message":"model is loading"}}. Switching to -sf is the right fix and the doc says why rather than only what.
The house form it cites confirms the recommendation rather than merely being mentioned: scripts/npu-window11.sh uses
curl -sf -m 2 "localhost:$PORT/health" >/dev/null 2>&1 || { echo "serve never became ready" >&2; exit 1; }
i.e. -sf against /health, exactly what the page now tells a reader to write. A doc that points at a script doing the thing it recommends is worth more than a doc that points at a script.
Two small notes, neither a change:
- the 503 you saw is
refuse(503, "no decision head loaded: …")inforward_serve.cpp— "while loading" is the effect, "no head yet" is the cause, and the page could say the second if it ever needs to be precise about which 503. curl -sfalso swallows a connection refused, which is what you want for a readiness loop and would be wrong for a diagnostic — the page is a readiness page, so it is right here, but that is the distinction a reader will hit next.
|
Persistent review updated to latest commit 4d992f5 |
bong-water-water-bong
left a comment
There was a problem hiding this comment.
VERDICT: READY at 4d992f5 — a main-merge, with the PR's own diff unchanged, so my previous review stands and I checked that rather than assuming it:
origin/main...4d1c316 docs/npu.md | 5 insertions, 1 deletion (the head I readied)
origin/main...4d992f5 docs/npu.md | 5 insertions, 1 deletion (this head: `Merge branch 'main' into lane/probe-doc`)
Same five added lines, same claim, and the claim still holds on today's main: no /api/v1/* route exists (serve.cpp, forward_serve.cpp and embed_serve.cpp register /health and /v1/health), curl -s exits 0 on a 404 so the probe "passes" against nothing, and the house script this page points at — scripts/npu-window11.sh — does curl -sf -m 2 …/health || exit 1, which is exactly what it now tells a reader to write.
A merge commit should not need a new reading, and this one does not. The two notes from that review stand and neither is a change: the 503's cause is "no decision head loaded" (the page says "while loading", the effect), and curl -sf also swallows connection-refused — right for a readiness loop, wrong for a diagnostic, and this is a readiness page.
Found by running the NPU fast lane on Qwen3-0.6B and writing the readiness probe from Lemonade's API shape:
So the readiness check passed while the weights were still loading, and the first chat request came back
{"error":{"message":"model is loading"}} - which I then compensated for with retries, hiding the real bug. With
-fand the right path the same smoke answers on the first request.The page already said
/v1/healthis "503 while loading"; it did not say there is no/api/v1/*prefix, which iswhat a lane copying the other API gets.
scripts/npu-window11.sh:430has the right form (curl -sf ... /health) -this sentence is the one-line version of why that form is right.
Additive only: 4 lines +1 reflowed, no headings touched,
doc_section_loss_check-> all 644 section(s) present,copyright --check rc 0.