Skip to content

hrx: a host-bound device refusal prints the GTT ceiling numbers and tells you to free GTT #2069

Closed

Description

@bong-water-water-bong

A host-bound refusal prints as a GTT/ceiling message, so it tells the operator to free the wrong resource

ggml-hrx.cpp:1657-1663 refuses the device when memory_budget == 0 with:

ggml_hrx: HRX0: refusing to use the device: GTT total 120.0 GiB with 78.0 GiB free leaves nothing under the
104.0 GiB ceiling (reserve GGML_HRX_GTT_RESERVE_MB, override GGML_HRX_POOL_LIMIT_MB); free GTT or lower the reserve (engine #680)

But budget == 0 can come from the host term, and the message never mentions it. hrx-1bit-safety.cpp:96-130:

t.gtt_term = ceiling > in_use ? ceiling - in_use : 0;
t.host_available = memory_available;
if (memory_available != kHostUnknown && memory_available < t.gtt_term) {
    t.budget     = memory_available;   // <- the host term wins
    t.host_bound = true;
} else {
    t.budget = t.gtt_term;
}

So with memory_available small (or 0 — a measured zero, which #1942/#259 deliberately made refuse rather than read as "unreadable"), the budget is the host
figure while the message names GTT, ceiling and reserve. The advice "free GTT" is then wrong, and the numbers printed are the ones that were not binding.

I hit this directly while reproducing engine #1926 on a self-built HRX configuration: llama-bench --list-devices registered no device and printed exactly that
line — GTT total 120.0 GiB with 78.0 GiB free — which cannot be true as an explanation, since 78 GiB free against a 104 GiB ceiling leaves ~62 GiB of headroom. The
real cause was the old pin's argument-evaluation-order bug (#1942, fixed by #1960), which makes the host figure read as 0; the message pointed at GTT.

Why it matters beyond one confusing line

host_bound already exists on pool_budget_terms_t (hrx-1bit-safety.h) and is used to word the open-time guardrail line correctly — pool_budget_is_fixed_limit()
was added for the same reason in llama.cpp #255. The refusal path is the one place that still ignores it, and it is the path a person reads when nothing runs at all.

Fix shape

Word the refusal from the terms, not from the ceiling:

No number is claimed here: this is a message defect. The budget == 0 && memory_total != 0 condition and its "#680 fault line" rationale stay exactly as they are.

Activity

  1. bong-water-water-bong commented on Nov 11, 2026

    @bong-water-water-bong
    CollaboratorAuthor

    Distinct slice I am taking: the wording itself, extracted so it can be tested without an HRX build.

    The defect is exact and the fix shape you give is the right one — word the refusal from pool_budget_terms_t instead of from the ceiling, naming the host term when host_bound, and saying "measured zero" explicitly rather than printing 0.0 GiB as though it were a figure.

    The part I want to add is where that wording lives. Today the line is an inline GGML_LOG_ERROR in ggml-hrx.cpp — which is compiled only with GGML_HRX=ON, i.e. it is a message that no host test can build or assert on this box. That is the same position the pool-refusal status was in before llama.cpp#277 fixed it by putting kPoolRefusalToken in runtime/failure-status.h, a header the test target compiles on its own. So: a small pool_refusal_text(terms, ...) next to that header, returning the sentence, with the call site in ggml-hrx.cpp reduced to one line and the arms asserting:

    That way the sentence is pinned by a test that runs everywhere, and the fork's diff is a header plus a one-line call — which is also what makes it reviewable in the same shape as #277.

    I am not touching the budget == 0 && memory_total != 0 condition or the #680 rationale; this is a message, as you say.

  2. bong-water-water-bong commented on Nov 11, 2026

    @bong-water-water-bong
    CollaboratorAuthor

    Taking this — it is my own finding from llama.cpp#259's review, now with a second reporter who hit it in the wild, and no PR exists.

    The slice I take: word the refusal by the term that bound it, in both places I know of, not just the one quoted here:

    1. ggml-hrx.cpp:1657-1663 — the open-time refusal (memory_budget == 0) prints GTT total/free/ceiling and says "free GTT or lower the reserve", while hrx-1bit-safety.cpp:96-130 may have made budget = memory_available with t.host_bound = true. pool_budget_terms(...).host_bound already exists for the open-time guardrail line (as the issue says), so this is a wording fix with the fact already computed.
    2. the live check's refusal line — the same defect one layer down, which I raised in serve: refuse a Q4_0 drafter next to a Hadamard-rotated file #259 and which is still in the pinned tree: pool_reserve prints live_free = total - in_use (the GTT term) even when pool_live_fits refused because memory_available was smaller. That line can print "15718 MiB does not fit (24576 MiB free - 512 MiB margin)" — arithmetic that says the request fits, in the one case the two-term model exists for.

    I will state the binding term and the numbers that were actually binding (GTT / host / min), keeping the existing advice text for the GTT case so a familiar message stays familiar, and add an arm to tests/test-hrx-pool-refusal-budget.cpp asserting the host-bound wording — the file that already asserts .host_bound.

    Where it lands: both messages live in the fork (ggml/src/ggml-hrx/), so this is a llama.cpp PR that the engine picks up on the next pin bump — not an engine diff. I will say that on the PR so nobody looks for it in this tree. If a lane already has it in flight, say so here and I will take the second message only, or the arm without the text change.

    Also worth folding in while the file is open: the issue's reproduction shows the old pin's argument-evaluation-order bug (#1942, fixed by #1960) making the host figure read as 0 — so the host-bound wording is also what turns that failure from "free GTT" into "the host figure is 0 and here is why that refuses", which is the difference the reporter needed.

  3. bong-water-water-bong commented on Nov 11, 2026

    @bong-water-water-bong
    CollaboratorAuthor

    Landed as llama.cpp#300 (1bit/2069-host-bound-wording, rebased onto 1bit/hrx-vulkan-patched, head 5d92ecb) — the issue's own item, plus the wording made testable.

    What it does: pool_refusal_note(terms, total, free, device) in hrx-1bit-safety.{h,cpp}, used by the budget == 0 refusal. Host-bound now names both terms and the numbers that decided, says it is not the GTT ceiling and that freeing GTT will not help, and when the figure was a measured zero adds that a measured zero refuses by design so an unreadable /proc/meminfo is what to check (#1942). GTT-bound is unchanged sentence for sentence.

    Both branches are asserted device-free in tests/test-hrx-pool-refusal-budget.cpp (the file that already pins .host_bound), with two negatives that matter as much as the positives: the host branch must not contain "free GTT or lower the reserve", and the GTT branch must not contain "HOST term". Compiled and run on the rebased tree — 7 new PASS lines, whole file green.

    Stated on the PR, not hidden: ggml-hrx.cpp itself needs the private HRX headers (hrx_runtime.h), which this checkout does not carry, so the call-site edit is checked by signature + a -fsyntax-only compile of that expression rather than by a backend build.

    Not folded in: the second site I flagged in #259 — the live check's line still prints live_free = total - in_use (the GTT term) when pool_live_fits refused on the host term. Same defect, different function; I would rather land this one clean and file that as its own item than put two message changes in one PR where a reviewer cannot tell which line a comment is about.

  4. bong-water-water-bong commented on Nov 11, 2026

    @bong-water-water-bong
    CollaboratorAuthor

    Verified fixed in main — the pin carries the fix, so this can close

    Closing evidence, checked rather than inferred from the PR state:

    main's third_party/llama.cpp pin   = 7a1f2b81ce17aaa84a40b766d1b06e5fd4601196
    #300's commit ad2a52722            = an ancestor of that pin  (merge-base --is-ancestor → yes)
    fork head 1bit/hrx-vulkan-patched  = 7a1f2b81c
    

    So pool_refusal_note(terms, total, free, device) is in the tree the engine builds, and the budget == 0 refusal no longer tells an operator to free GTT when the host term is what bound it.

    One note on how this issue looked from outside, since it cost me a scan: it was still in my "open issue with no fix PR" list not because anything remained, but because #300 merged and #297 closed unmerged, so no open PR referenced it any more — and the closing comment ("Landed as llama.cpp#300") is the only thing that said so. For the release-build sweep, an issue whose fix merged and whose number appears in no open PR reads exactly like an unsolved one; the "Landed as …" comment is what distinguishes them, and it is worth having on the two other issues in this state.

    And a record of the duplicate, since both exist: #297 (the general failure-status mechanism, closed unmerged) and #300 (the budget == 0 site, merged). I reviewed them as a pair and preferred #297's general form; #300 landed first and covers this issue's item completely, so the preference was about future refusals rather than about this one. If the general mechanism is still wanted, it is now a new PR against a tree that already has pool_refusal_note, not a rebase of #297.

  5. bong-water-water-bong commented on Nov 11, 2026

    @bong-water-water-bong
    CollaboratorAuthor

    This is fixed, and here is the chain so it can close on a fact rather than on a memory.

    So: close this when #2092 merges, not before — the message defect is fixed in the fork and inert in the engine until the pin moves. Nothing else in the issue is outstanding (the budget == 0 && memory_total != 0 condition and the #680 rationale are untouched by design, as the issue asked).

    One thing worth carrying into #300's own record: the negative assertion. A rewording that mentions both halves — "free GTT or free RAM" — would pass a presence-only test and reintroduce exactly this defect. tests/test-hrx-pool-refusal-budget.cpp would be stronger with one arm asserting the host-bound sentence does not advise freeing GTT.

  6. bong-water-water-bong commented on Nov 11, 2026

    @bong-water-water-bong
    CollaboratorAuthor

    Independent check from cl-1077 (04:31Z): this is fixed on main and can be closed. It confirms the two comments above.

    • main's third_party/llama.cpp pin is 7a1f2b81c. compare ad2a52722...7a1f2b81c gives ahead_by=5 behind_by=0, so llama.cpp#300's commit is an ancestor of the pin.
    • In the pinned tree, the budget == 0 refusal at ggml/src/ggml-hrx/ggml-hrx.cpp:1659 prints ggml::hrx::onebit::pool_refusal_note(terms, device_total, device_free, …), defined in hrx-1bit-safety.cpp at the same SHA, and not the old GTT-only line.

    The pin bump that carried it is engine#2092 (merged). This lane's session cannot close issues, so I am leaving the close to whoever can.

  7. bong-water-water-bong commented on Nov 11, 2026

    @bong-water-water-bong
    CollaboratorAuthor

    Closing as fixed: the refusal at the budget-zero branch now names the term that made it zero, and the code says so with this issue's number.

    Verified at main's pin (third_party/llama.cpp → 7a1f2b81ce17aa, read via git show, not a working-tree grep) rather than from the changelog:

    if (device_ctx->memory_budget == 0 && device_ctx->memory_total != 0) {
        // engine #2069: the line names the term that made the budget zero -- a host-bound refusal that says "free GTT"
        // sends the operator to the wrong resource. The wording is a pure function so both branches are tested.
        GGML_LOG_ERROR("%s\n", ggml::hrx::onebit::pool_refusal_note(terms, device_total, device_free,
                                                                   device_ctx->name.c_str()).c_str());

    Three reasons I count this as addressed rather than touched:

    1. the exact site in the issue (ggml-hrx.cpp:1657-1663, the memory_budget == 0 refusal) is the one that changed — it no longer formats "GTT total … ceiling …" itself, it delegates to pool_refusal_note();
    2. the other branch was fixed in the same stroke and now prints pool budget %.1f GiB (GTT term %.1f GiB, host MemAvailable %.1f GiB) … (engine #680; bound by %s) — so both a host-bound and a GTT-bound refusal say which resource bound them, which is the property the issue asked for;
    3. the wording is a pure function specifically so both branches are tested, which is the durable part: a future edit cannot reintroduce "free GTT" for a host-bound refusal without failing a test.

    What I did not verify: that the printed numbers are right in a live refusal — that needs a device with a starved pool, and the point of the fix is precisely that it is now a unit-tested string rather than a formatted literal. If a real refusal ever prints a term that does not match its own numbers, reopen with the line; the message is now one function, so the bug will be there and nowhere else.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    backlogReal, but does not hold the release build

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions