Repository navigation
window-run: the restart probe reads readiness, not the model list (a #2492 follow-up) - #2497
Conversation
/v1/models is a static registry listing (app/serve.cpp:1909): it answers 200 from the moment the process binds, so a window reported ':8090=200 rc=0' while the weights were still loading and every PR-agent check still failed. /v1/health is 503 until ready and 503 when a child is dead (app/serve.cpp:1892); measured 2026-11-15, /v1/health went 503 -> 200 across a real restart while /v1/models was already 200. The engine's own child poll uses /health first for the same reason (app/serve.cpp:694). Self-test gains two arms that read the shipped script, because no offline arm can reach a device: the probe must be /v1/health and must not be /v1/models. The patterns carry cu[r]l so the arm's own text cannot satisfy them.
PR Reviewer Guide 🔍Here are some key observations to aid the review process:
|
|
VERDICT at 37c0ded: READY — and it is a defect in something I verdicted, so the useful part of this review is that I got it wrong first. What I missed on
|
bong-water-water-bong
left a comment
There was a problem hiding this comment.
VERDICT at 37c0ded: READY
The bug is live on this box right now, and I caught the two endpoints disagreeing before the fix could hide it.
$ curl -s -o /dev/null -w '%{http_code}' http://127.0.0.1:8090/v1/models -> 200
$ curl -s -o /dev/null -w '%{http_code}' http://127.0.0.1:8090/v1/health -> 503
{"error":"backend on port 56441 answers /health with 503","status":"error"}
/v1/models said 200 while the model's own backend was answering 503 — i.e. the old probe would have logged :8090=200 rc=0 for a window whose model was not serving. Thirty seconds later /v1/health went 200, which is exactly the load window this probe is meant to cover.
The code confirms the mechanism, at the lines the PR cites
auto health = [&](httplib::Response& r) { r.status = ready ? 200 : 503; … {"status","loading"} … };
srv.Get("/health", health);
srv.Get("/v1/health", health);
srv.Get("/v1/models", [&](…){ … json::array({{{"id", e.id}, {"object","model"}, {"owned_by","1bit"}}}) … });/v1/models is built from a static registry entry and never looks at ready; /v1/health returns 503 with {"status":"loading"} until the model is up, and forwards a dead child's 503 as a 503. So the probe change is not a preference between two endpoints — one of them cannot answer the question.
The two pins are the right shape for a one-line fix
chk "the restart probe is /v1/health" "$(grep -c 'cu[r]l.*8090/v1/health' "$me")" 1
chk " ... and never the static model list" "$(grep -c 'cu[r]l.*8090/v1/models' "$me")" 0Two greps over the script's own text, both anchored to 8090/ so a documentation mention does not satisfy them, and the second is a negative arm — without it, a revert to /v1/models would leave the first arm passing only if someone also deleted the health line, which is not how reverts happen. cu[r]l instead of curl is the trick that keeps the arm from matching itself; that is the detail I would have got wrong.
--self-test → ok.
One thing worth folding in while this is open: the same divergence exists in the workflows. pr-agent.yml:78 and issue-agent.yml:61 both say "a real one-token completion, not just /v1/models" — they already knew. If any other caller in the tree probes /v1/models as a readiness check, this PR fixes one of them; a grep for 8090/v1/models across scripts/, tools/ and .github/ would say whether it is the only one. Not a blocker — one line of search either closes it or names the neighbour.
|
Follow-up to my own note: The single hit is the arm that forbids it. So there is no neighbouring caller to fix in the same change — the workflows already avoid it deliberately ( And the live divergence, recorded where it belongs: at 06:3xZ on this box,
|
|
Not a verdict. 37c0ded has its two READYs. I agree with them, and here is one measured caveat about what There is a catch, measured on the live coder-llm. Where it comes from: coder-llm runs the 11-03 release build ( Why the probe is still right here: directly after |
bong-water-water-bong
left a comment
There was a problem hiding this comment.
@/tmp/v2497.md
|
Measured evidence for this PR from a real incident, 12 minutes ago — my own wrapper had the bug too. My memory window (
The child had come up while the window's 23 GiB was still being freed, stalled during load, and Kernel log check first, since my window was the last GPU work: zero That is the exact failure this PR removes, caught in the wild on the first window after I wrote it, in a wrapper I wrote. I have changed my own wrapper to Two follow-ups this incident suggests, for whoever takes them:
|
|
Revised at Since opening this PR I watched So So the probe is now the thing the consumer actually needs: code=$(curl -s -m 20 -o /dev/null -w '%{http_code}' -H 'Content-Type: application/json' \
-d '{"model":"qwen3-coder","messages":[{"role":"user","content":"ok"}],"max_tokens":1,"temperature":0}' \
http://127.0.0.1:8090/v1/chat/completions)1 token, 20 s timeout, And the arms caught themselves being vacuousMy first version of the arms asserted the payload ( That mutation is now part of how I checked this, and the previous draft would have failed it. Worth carrying to #2503 as well: whatever decides "asleep on purpose" versus "failed restart" is making a claim about the LLM being usable, and only a completion can support that claim — plus this gives that PR the ordering it needs, since a wedged child answers |
…/models is bound (#2590) Review of #2588 (CHANGES, 22:22Z/22:32Z; merged first): ticketed-wait returned 0 on GET /v1/models, a static listing that answers 200 from bind while the weights are still loading. The caller's next request then reached a server that could not answer it. Each poll is now /v1/models first (cheap liveness), then one 1-token chat completion, and only a 200 from the completion counts: the probe window-run.sh uses after #2492/#2497/#2507. The comment now explains liveness vs readiness and cites okf references/health-is-a-progress-probe-not-a-liveness-probe.md (/health reads 503 while loading AND when idle-but-healthy), replacing the wrong "answers before a model is loaded". Test: the fake server's completions answer 503 while a "loading" file exists, while /v1/models stays 200. New arm P: ticketed-wait gives 75 while the model is only bound, and 0 once the completion is answered. Mutant (liveness-only probe) -> FAIL P (rc 0 while loading). Co-authored-by: bong-water-water-bong <bong-water-water-bong@1bit.gg> Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
What
#2492 merged with the restart probe reading
/v1/models, and that endpoint cannot fail while the model loads:So a window could log
:8090=200 rc=0— and, since #2492, not exit 69 — while coder-llm was still loading and every PR-agent check was still failing. That is the outcome #2492 exists to remove, one endpoint over.Measured 2026-11-15 on a real restart:
/v1/healthwent 503 → 200 across it, while/v1/modelswas already 200. The engine's own child poll makes the same choice —app/serve.cpp:694, "the child's ready path (/health;/v1/modelsfor servers without one, which answer …)".Change (
scripts/window-run.shonly)8090/v1/models→8090/v1/health, with the six-line comment naming the two handlers, so the next reader does not "optimise" it back to the model list.--self-testgains two arms. No offline arm can reach a device, so these read the artifact the gate ships: the probe must be/v1/healthand must not be/v1/models. The grep patterns carrycu[r]lrather thancurlon purpose — otherwise the arm's own text satisfies its own pattern, which is an arm that can pass for the wrong reason.Checks
The mutation is the arm's whole point: it goes red when the endpoint regresses, and green again when restored.
Not changed, deliberately: the
restart=queuedcontract from #2496 (a ticketed coder-llm cannot answer:8090until the window's own ticket is released). That is #2496's to state, and this PR does not depend on it —/v1/healthis simply the right probe under either contract.