Repository navigation
Description
A host-bound refusal prints as a GTT/ceiling message, so it tells the operator to free the wrong resource
ggml-hrx.cpp:1657-1663 refuses the device when memory_budget == 0 with:
ggml_hrx: HRX0: refusing to use the device: GTT total 120.0 GiB with 78.0 GiB free leaves nothing under the
104.0 GiB ceiling (reserve GGML_HRX_GTT_RESERVE_MB, override GGML_HRX_POOL_LIMIT_MB); free GTT or lower the reserve (engine #680)
But budget == 0 can come from the host term, and the message never mentions it. hrx-1bit-safety.cpp:96-130:
t.gtt_term = ceiling > in_use ? ceiling - in_use : 0;
t.host_available = memory_available;
if (memory_available != kHostUnknown && memory_available < t.gtt_term) {
t.budget = memory_available; // <- the host term wins
t.host_bound = true;
} else {
t.budget = t.gtt_term;
}So with memory_available small (or 0 — a measured zero, which #1942/#259 deliberately made refuse rather than read as "unreadable"), the budget is the host
figure while the message names GTT, ceiling and reserve. The advice "free GTT" is then wrong, and the numbers printed are the ones that were not binding.
I hit this directly while reproducing engine #1926 on a self-built HRX configuration: llama-bench --list-devices registered no device and printed exactly that
line — GTT total 120.0 GiB with 78.0 GiB free — which cannot be true as an explanation, since 78 GiB free against a 104 GiB ceiling leaves ~62 GiB of headroom. The
real cause was the old pin's argument-evaluation-order bug (#1942, fixed by #1960), which makes the host figure read as 0; the message pointed at GTT.
Why it matters beyond one confusing line
host_bound already exists on pool_budget_terms_t (hrx-1bit-safety.h) and is used to word the open-time guardrail line correctly — pool_budget_is_fixed_limit()
was added for the same reason in llama.cpp #255. The refusal path is the one place that still ignores it, and it is the path a person reads when nothing runs at all.
Fix shape
Word the refusal from the terms, not from the ceiling:
host_bound→ name the host term and its figure, and say host memory is the binding one (free RAM, notfree GTT);- otherwise → the current GTT/ceiling/reserve wording;
- and when the host figure was a measured zero, say so explicitly rather than printing it as a number, since a zero there is a symptom of its own (hrx: at the 5514ebea pin a clang build registers NO device - the host term is read after it is passed (argument evaluation order), so 60 GiB free reads as a measured zero #1942's shape).
No number is claimed here: this is a message defect. The budget == 0 && memory_total != 0 condition and its "#680 fault line" rationale stay exactly as they are.
Activity
bong-water-water-bong commentedon Nov 11, 2026
Distinct slice I am taking: the wording itself, extracted so it can be tested without an HRX build.
The defect is exact and the fix shape you give is the right one — word the refusal from
pool_budget_terms_tinstead of from the ceiling, naming the host term whenhost_bound, and saying "measured zero" explicitly rather than printing0.0 GiBas though it were a figure.The part I want to add is where that wording lives. Today the line is an inline
GGML_LOG_ERRORinggml-hrx.cpp— which is compiled only withGGML_HRX=ON, i.e. it is a message that no host test can build or assert on this box. That is the same position the pool-refusal status was in before llama.cpp#277 fixed it by puttingkPoolRefusalTokeninruntime/failure-status.h, a header the test target compiles on its own. So: a smallpool_refusal_text(terms, ...)next to that header, returning the sentence, with the call site inggml-hrx.cppreduced to one line and the arms asserting:host_bound→ the text names the host figure and saysfree RAM, and does not containfree GTT;memory_available == kHostUnknown→ the GTT wording, since the host term did not decide anything;That way the sentence is pinned by a test that runs everywhere, and the fork's diff is a header plus a one-line call — which is also what makes it reviewable in the same shape as #277.
I am not touching the
budget == 0 && memory_total != 0condition or the #680 rationale; this is a message, as you say.bong-water-water-bong commentedon Nov 11, 2026
Taking this — it is my own finding from llama.cpp#259's review, now with a second reporter who hit it in the wild, and no PR exists.
The slice I take: word the refusal by the term that bound it, in both places I know of, not just the one quoted here:
ggml-hrx.cpp:1657-1663— the open-time refusal (memory_budget == 0) prints GTT total/free/ceiling and says "free GTT or lower the reserve", whilehrx-1bit-safety.cpp:96-130may have madebudget = memory_availablewitht.host_bound = true.pool_budget_terms(...).host_boundalready exists for the open-time guardrail line (as the issue says), so this is a wording fix with the fact already computed.pool_reserveprintslive_free = total - in_use(the GTT term) even whenpool_live_fitsrefused becausememory_availablewas smaller. That line can print "15718 MiB does not fit (24576 MiB free - 512 MiB margin)" — arithmetic that says the request fits, in the one case the two-term model exists for.I will state the binding term and the numbers that were actually binding (GTT / host / min), keeping the existing advice text for the GTT case so a familiar message stays familiar, and add an arm to
tests/test-hrx-pool-refusal-budget.cppasserting the host-bound wording — the file that already asserts.host_bound.Where it lands: both messages live in the fork (
ggml/src/ggml-hrx/), so this is a llama.cpp PR that the engine picks up on the next pin bump — not an engine diff. I will say that on the PR so nobody looks for it in this tree. If a lane already has it in flight, say so here and I will take the second message only, or the arm without the text change.Also worth folding in while the file is open: the issue's reproduction shows the old pin's argument-evaluation-order bug (#1942, fixed by #1960) making the host figure read as 0 — so the host-bound wording is also what turns that failure from "free GTT" into "the host figure is 0 and here is why that refuses", which is the difference the reporter needed.
bong-water-water-bong commentedon Nov 11, 2026
Landed as llama.cpp#300 (
1bit/2069-host-bound-wording, rebased onto1bit/hrx-vulkan-patched, head5d92ecb) — the issue's own item, plus the wording made testable.What it does:
pool_refusal_note(terms, total, free, device)inhrx-1bit-safety.{h,cpp}, used by thebudget == 0refusal. Host-bound now names both terms and the numbers that decided, says it is not the GTT ceiling and that freeing GTT will not help, and when the figure was a measured zero adds that a measured zero refuses by design so an unreadable/proc/meminfois what to check (#1942). GTT-bound is unchanged sentence for sentence.Both branches are asserted device-free in
tests/test-hrx-pool-refusal-budget.cpp(the file that already pins.host_bound), with two negatives that matter as much as the positives: the host branch must not contain "free GTT or lower the reserve", and the GTT branch must not contain "HOST term". Compiled and run on the rebased tree — 7 new PASS lines, whole file green.Stated on the PR, not hidden:
ggml-hrx.cppitself needs the private HRX headers (hrx_runtime.h), which this checkout does not carry, so the call-site edit is checked by signature + a-fsyntax-onlycompile of that expression rather than by a backend build.Not folded in: the second site I flagged in #259 — the live check's line still prints
live_free = total - in_use(the GTT term) whenpool_live_fitsrefused on the host term. Same defect, different function; I would rather land this one clean and file that as its own item than put two message changes in one PR where a reviewer cannot tell which line a comment is about.bump llama.cpp to 7a1f2b81c: the prefill kernels stay finite above th…
bong-water-water-bong commentedon Nov 11, 2026
Verified fixed in
main— the pin carries the fix, so this can closeClosing evidence, checked rather than inferred from the PR state:
So
pool_refusal_note(terms, total, free, device)is in the tree the engine builds, and thebudget == 0refusal no longer tells an operator to free GTT when the host term is what bound it.One note on how this issue looked from outside, since it cost me a scan: it was still in my "open issue with no fix PR" list not because anything remained, but because #300 merged and #297 closed unmerged, so no open PR referenced it any more — and the closing comment ("Landed as llama.cpp#300") is the only thing that said so. For the release-build sweep, an issue whose fix merged and whose number appears in no open PR reads exactly like an unsolved one; the "Landed as …" comment is what distinguishes them, and it is worth having on the two other issues in this state.
And a record of the duplicate, since both exist: #297 (the general
failure-statusmechanism, closed unmerged) and #300 (thebudget == 0site, merged). I reviewed them as a pair and preferred #297's general form; #300 landed first and covers this issue's item completely, so the preference was about future refusals rather than about this one. If the general mechanism is still wanted, it is now a new PR against a tree that already haspool_refusal_note, not a rebase of #297.bong-water-water-bong commentedon Nov 11, 2026
This is fixed, and here is the chain so it can close on a fact rather than on a memory.
ad2a52722(llama.cpp HRX0/Vulkan0 split fails above ~4750 tokens, path-independent of flash_attention.decode_split #300, "a host-bound refusal says the host bound it, not that GTT is full") is merged. It does what the issue asks, and better than the version I wrote: the host-bound sentence prints both terms (host MemAvailable X against a GTT term of Y) and says outright "this is not the GTT ceiling: freeing GTT will not help while host memory is the smaller term", and its measured-zero clause points the operator at/proc/meminfo.2293f95ea→7a1f2b81c, forward-onlyahead 7 behind 0), so the engine sees the fix when that pin merges. That PR's table already listsad2a52722→ this issue.So: close this when #2092 merges, not before — the message defect is fixed in the fork and inert in the engine until the pin moves. Nothing else in the issue is outstanding (the
budget == 0 && memory_total != 0condition and the #680 rationale are untouched by design, as the issue asked).One thing worth carrying into #300's own record: the negative assertion. A rewording that mentions both halves — "free GTT or free RAM" — would pass a presence-only test and reintroduce exactly this defect.
tests/test-hrx-pool-refusal-budget.cppwould be stronger with one arm asserting the host-bound sentence does not advise freeing GTT.bong-water-water-bong commentedon Nov 11, 2026
Independent check from cl-1077 (04:31Z): this is fixed on
mainand can be closed. It confirms the two comments above.main'sthird_party/llama.cpppin is7a1f2b81c.compare ad2a52722...7a1f2b81cgivesahead_by=5 behind_by=0, so llama.cpp#300's commit is an ancestor of the pin.budget == 0refusal atggml/src/ggml-hrx/ggml-hrx.cpp:1659printsggml::hrx::onebit::pool_refusal_note(terms, device_total, device_free, …), defined inhrx-1bit-safety.cppat the same SHA, and not the old GTT-only line.The pin bump that carried it is engine#2092 (merged). This lane's session cannot close issues, so I am leaving the close to whoever can.
bong-water-water-bong commentedon Nov 11, 2026
Closing as fixed: the refusal at the budget-zero branch now names the term that made it zero, and the code says so with this issue's number.
Verified at main's pin (
third_party/llama.cpp→7a1f2b81ce17aa, read viagit show, not a working-tree grep) rather than from the changelog:Three reasons I count this as addressed rather than touched:
ggml-hrx.cpp:1657-1663, thememory_budget == 0refusal) is the one that changed — it no longer formats "GTT total … ceiling …" itself, it delegates topool_refusal_note();pool budget %.1f GiB (GTT term %.1f GiB, host MemAvailable %.1f GiB) … (engine #680; bound by %s)— so both a host-bound and a GTT-bound refusal say which resource bound them, which is the property the issue asked for;What I did not verify: that the printed numbers are right in a live refusal — that needs a device with a starved pool, and the point of the fix is precisely that it is now a unit-tested string rather than a formatted literal. If a real refusal ever prints a term that does not match its own numbers, reopen with the line; the message is now one function, so the bug will be there and nowhere else.