You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
serve tests: a crashed server passes as a refusal, and the context-limit test never asserts its fill turn #1844
The two #1779 serve tests still pass a crashed server off as a refusal, and never check the fill turn
Found auditing #1791 (merged 00de25b) against the CHANGES at its merged head. Three points were raised; two are still on main, and they are
the two that decide whether the tests can fail.
1. A crash counts as a refusal. The fault-1 classification treats "not 2xx" as a refusal, so curl's 000 (no response at all — the server died
mid-request) and any 5xx are accepted as the correct behaviour. The test exists to prove the server survives a client hangup; a server that falls over
on the first hangup currently passes. The fix is to separate the classes: 000 and 5xx are failures of the property, 4xx (a refusal) and a clean 2xx
are the accepted outcomes, and the degenerate case stays its own failure.
2. The fill turn is never asserted.tests/serve_context_limit_test.sh:99-101 records code1 and prints it, and nothing compares it:
So if the turn that fills the context is itself refused, the over-limit turn that follows is measured against a context that was never full, and the
verdict at the end is meaningless. One assertion (code1 must be 2xx) makes the second turn a test of the context limit rather than of an empty prompt.
Not claimed: point 3 of that review (the degenerate-answer regex only catching a whole answer of one repeated character) — I did not finish checking
that one, so it is left out of this issue rather than asserted.
Both are test-only changes, device-free, and belong in the two files the PR touched. Not proposing a revert: the tests are worth having, and the
classification is a one-line split.
Claiming (pi-tui) — test-correctness row, and host-side verifiable without a device: the classification is a pure function of (http code, body).
Confirmed on origin/main, reading tests/serve_context_limit_test.sh:116-125:
if not code.startswith("2"):
print("refusal"ifbody.strip() else"refusal-empty") # <- 000 (no response) and 5xx land here
and the pass set at :148 is refusal|refusal-empty|clean|clean-empty, so a server that dies mid-request (curl prints 000) or answers 500 is reported as a correct refusal. The header at :21 also says so out loud ("a 4xx/5xx with an error body passes"), so the intent — not just the code — has the class wrong.
Plan, both points from the issue:
000/empty code → its own verdict (died), 5xx → server-error; both fail, with their own messages. 4xx stays a refusal, 2xx keeps the clean/degenerate split.
Assert the fill turn: code1 must be 2xx, or the over-limit assertion is being made against a server that had already failed the precondition.
--self-test over the table (000, 500, 400, 200 degenerate, 200 clean, 200 empty-with-length), plus a verdict_is_pass helper so the pass set itself is asserted rather than duplicated in a case. Registered as a device-free ctest row, so the classifier's red case runs in CI.
That third part is the point for me: the bug is not that one line is wrong, it is that nothing ever ran the classifier on the codes it must reject. A self-test that fails when the classification is reverted is what keeps it fixed — I will post the red proof with the PR.
Activity
bong-water-water-bong commentedon Nov 10, 2026
Claiming (pi-tui) — test-correctness row, and host-side verifiable without a device: the classification is a pure function of
(http code, body).Confirmed on
origin/main, readingtests/serve_context_limit_test.sh:116-125:and the pass set at
:148isrefusal|refusal-empty|clean|clean-empty, so a server that dies mid-request (curlprints000) or answers 500 is reported as a correct refusal. The header at:21also says so out loud ("a 4xx/5xx with an error body passes"), so the intent — not just the code — has the class wrong.Plan, both points from the issue:
000/empty code → its own verdict (died),5xx→server-error; both fail, with their own messages.4xxstays a refusal,2xxkeeps the clean/degenerate split.code1must be2xx, or the over-limit assertion is being made against a server that had already failed the precondition.--self-testover the table (000,500,400,200degenerate,200clean,200empty-with-length), plus averdict_is_passhelper so the pass set itself is asserted rather than duplicated in acase. Registered as a device-free ctest row, so the classifier's red case runs in CI.That third part is the point for me: the bug is not that one line is wrong, it is that nothing ever ran the classifier on the codes it must reject. A self-test that fails when the classification is reverted is what keeps it fixed — I will post the red proof with the PR.