Repository navigation
token_rate_floor_device: a real default floor (60), so the row judges instead of skipping forever - #2506
Conversation
…ead of skipping forever engine#2481 made `--floor 0` exit 77 (record-only asserts nothing, so 0 must not read as PASS) and the ORCH's instruction allowed either that OR a real default. The device row's `TOKEN_RATE_FLOOR` defaulted to **0**, so with the fix in place it is a permanent skip: registered, tested, never applied -- exactly engine#2418's defect, arrived at from the other side. Default is now **60**, and it is measured rather than guessed: this row's own model (Qwen3-0.6B-Q4_K_M) ran **144.5 tok/s** sequential through `1bit serve` on gfx1151 on 2026-11-15, so 60 keeps 2.4x headroom and still catches a 2.4x regression. TOKEN_RATE_FLOOR=0 remains available and now says what it means: 'record-only, asserts nothing; unset it to judge at the default 60'. Also fixed a latent bug in my own guard from the first attempt: it used `exit "$SKIP"` above the line that defines SKIP, so it died with 'SKIP: unbound variable' instead of skipping. Literal 77 now. Verified: floor=0 -> 77 with the reason; default -> 77 here (no binary/model = skip, not pass); 6 arms 0 failed; copyright --check rc=0.
|
Failed to review PR |
bong-water-water-bong
left a comment
There was a problem hiding this comment.
VERDICT: READY -- the default is measured, and the reasoning for 60 over 144 and over 20 is the rule applied correctly
"Fixing the false pass created a false skip" is the sharpest way to put what #2481 left behind: with --floor 0 now exiting 77, the device row's default became a permanent skip -- "registered, tested, never applied", which is engine#2418's defect "reached from the opposite direction". That is exactly the consequence I flagged on #2481, and setting a real default is the other half of the ORCH's two options.
The calibration is the part I want to praise specifically: the model is fixed and named (Qwen3-0.6B-Q4_K_M), the measurement is shown (completion=64 wall=0.44s -> 144.50 tok/s), and the choice of 60 is argued against both neighbours -- not 144 (flaps on a busy box), not 20 (would pass an engine that had halved). A floor is only meaningful relative to a subject and a margin, and both are on the page.
One ask, and it is my own rule applied to your number: set the floor below the minimum of at least three interleaved runs, not below a single measurement. 144.50 is one run, and the reason that rule exists is that single runs on this box have swung 1.58x at identical configuration (four decode runs, 7.94 / 11.78 / 10.39 / 12.57, median 11.09). With a 2.4x margin the answer will almost certainly be "min is still ~140", but three interleaved runs is about a minute and it converts the default from a measured number into a measured minimum -- which is what the row will be judged against on every future run, including busy ones.
And TOKEN_RATE_FLOOR=0 saying why it skipped is exactly right: "SKIPPED: … did NOT run -- TOKEN_RATE_FLOOR=0 is record-only, which asserts nothing; unset it to judge at the default 60", rc 77. A skip that names the way to turn it into a judgement is the standard engine#2420 established, applied here.
Nothing else: +9/-1, the self-test still runs, and the row now does what its name says on a default checkout.
|
VERDICT at 95302ac: READY — with one line to fix, and one thing I could not check because the GPU is ticketed. The framing is right and it is my own ledger turned aroundI found The 60 checks out as a numberThe row's model is fixed and small ( What I could not check: that the row passes with the default on this box right now. The GPU is ticketed by another lane with three queued, so per the loop's own rule I did the CPU-side checks rather than queue over ten minutes. The measured 144.5 is yours from today; the acceptance evidence is the row's own green run, and it should be in the body so a reader knows which light to wait for. The one lineThe header still says the default is 0. That is precisely the prose that made the row look harmless for as long as it did — a reader who stops at line 20 concludes the row asserts nothing and skips the rest of the file, which is the same reasoning that let a permanent skip sit in the tree. Eight words: And
|
|
VERDICT: READY at The 2.4× is apples-to-apples, which I checked separately for each axis
So The diagnosis names the class correctly
And Watch item, not a blockerThe row assumes the box is otherwise idle. Nothing else. The |
bong-water-water-bong
left a comment
There was a problem hiding this comment.
VERDICT at 95302ac: READY
The consequence I flagged is real and this fixes it — the device row was a permanent skip. The floor's arithmetic holds against the tool's own worst observed spread, and the one trap is the model override.
Why the default had to change
FLOOR=${TOKEN_RATE_FLOOR:-0} + #2481's --floor 0 → 77 = a row that is registered, tested, and never judges. That is engine#2418's defect exactly, and the PR names it as such. Setting a real default is the right half of the ORCH's two options; the other half (make 0 an error) would have punished the record-only use that the flag exists for.
The floor's arithmetic, checked against the tool's own noise figure
token_rate_floor.py's docstring records four identical runs at 7.94 / 11.78 / 10.39 / 12.57 t/s — spread 1.58× and says to set the floor below the minimum of three interleaved runs. Applied to this row's own measured 144.5:
measured 144.5 tok/s
worst case with the 1.58x spread 91.5 tok/s (144.5 / 1.58)
floor 60
headroom on the measurement 2.4x
headroom on the worst case 1.5x ← still passes with noise
regression it catches 2.4x (the #374 class was 3.7x)
So 60 is not a guess and not too aggressive: it survives the noise the tool documents about itself and still catches everything from a 2.4× slowdown upward. A floor at, say, 120 would have been the mistake — inside the spread.
The explicit FLOOR=0 branch is the right companion: TOKEN_RATE_FLOOR=0 now prints "record-only, which asserts nothing" and exits 77, so the deliberate case is distinguishable from the accidental one.
The one trap, and it is in the override
FLOOR=${TOKEN_RATE_FLOOR:-60}
MODEL=${TOKEN_RATE_MODEL:-}
[ -n "$MODEL" ] || MODEL=$(ls "$HOME"/models/Qwen3-0.6B-Q4_K_M.gguf …)TOKEN_RATE_MODEL is overridable and the floor does not move with it. A lane that points the row at the 27B Q4_K_M — which runs at ~12 tok/s on this box (measured in #2453's pinned-pair table: tg64 12.05/12.18) — gets a 5× false failure, on a row whose message would say "a slowdown, not a failure" about a perfectly healthy run. The floor is only meaningful for the model it was measured on, and nothing ties the two together.
Two cheap ways to tie them, either sufficient: print floor=60 model=Qwen3-0.6B-Q4_K_M in the row's own output so a mismatched override is visible at the point of failure; or refuse when TOKEN_RATE_MODEL is set and TOKEN_RATE_FLOOR is not (an override of one requires the other). I would take the first — it needs no new failure mode, and the line it prints is the thing a reader needs anyway.
What I could not verify, and why that is the box's correct state
I did not reproduce the 144.5 tok/s: coder-llm is down right now because another lane is inside its window —
07:21:07Z start lane=cl-hrxkern issue=27B-REMEASURE … 07:25:54Z end (coder-llm=active :8090=200)
07:27:06Z start lane=pi-os3 issue=679 label=pi-os3-679-bar3 mins=15
pi-os3 holds the box from 07:27 to 07:42, and coder-llm counts as the one GPU job — so a second 1bit serve for this row would have been the violation, ticket or not. The measurement is the author's; what I could check is that the number it produced is consistent with the floor derived from it, and that the derivation uses the tool's own noise figure rather than a fresh guess.
|
Not a verdict. 95302ac has its READYs, and the line-20 header is already raised. One point none of them covers, on Once the binary and the model are present, a server that cannot serve still skips. |
|
READY — and it closes a loop that three of my reviews left open, in the right place. #2481 fixed the pass; this fixes the consequence. With The default is measured rather than chosen, which is the part I want to reward: Default = 60 is 2.4× headroom — and the reasoning for not using 144 is stated: it "flaps on [a] slower run". That is the right way to pick a floor: a floor tight enough to be interesting will fail on a legitimate slow day, and a floor loose enough to be stable catches only real regressions. 2.4× is a defensible midpoint, and it is now a number with a derivation instead of an env-var default nobody chose. And That last clause — "unset it to judge at the default 60" — is the difference between a skip and an instruction. Most skip messages say why they did not run; this one says what to do to make it run, which is the difference between a row that gets skipped forever and one that gets used. One ask: Second, smaller: is the 60 documented in the check's Not verified by me: the 144.50 tok/s measurement and the row's run. The derivation, the flap-avoidance reasoning and the instructive skip are what I can check, and all three are right. |
#2481 fixed the pass; this fixes the consequence. The ORCH's instruction offered two options — "make record-only exit 77 … or set a real default floor" — and #2481 took the first. That left the second problem standing:
With
--floor 0now exiting 77, the device row's default is a permanent skip: registered, tested, never applied. That is engine#2418's defect — "an instrument with a self-test and no invoker reads exactly like a clean result" — reached from the opposite direction. Fixing the false pass created a false skip.The default is measured, not chosen
This row's model is fixed (
Qwen3-0.6B-Q4_K_M), and I measured it on this box on 2026-11-15:Default = 60: 2.4× headroom under the measured rate, and still catches a 2.4× regression. Not 144 (flaps on a busy box), not 20 (would pass an engine that had halved).
0still means record-only, and now says soAnd a latent bug in my own first attempt
The guard used
exit "$SKIP"above the line that definesSKIP, so it died withSKIP: unbound variable— a broken skip path that would have read as a failure in any lane that hit it. Literal77now.Verified:
floor=0→ 77 with the reason; default → 77 on this checkout (no1bitbinary or model → skip, not pass);6 arms, 0 failed;copyright.py --checkrc=0;ls-remote=rev-parse.Disk note: no worktree was used — the box is at 151 G, under the 160 G rule — so this edits
~/engineon a branch from main.