You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
hrx: plain (non-fused) MUL_MAT_ID on bf16 weights returns NaN for activations above f16's range, and #2127/#309 does not cover it #2167
Split out of #2127 after measuring, because the fix that issue produced (#309) touches a different kernel and I could confirm that statically from its diff.
Measured
test-backend-ops -b HRX0 -o MUL_MAT_ID at pin 7a1f2b81ce17aa, one row changed between the arms:
MUL_MAT_ID(type_a=bf16,type_b=f32,n_mats=4,n_used=2,m=512,n=32,k=256,activation_scale=10000.000000): OK
MUL_MAT_ID(type_a=bf16,type_b=f32,n_mats=4,n_used=2,m=512,n=32,k=256,activation_scale=100000.000000):
[MUL_MAT_ID] NaN at index 0 (HRX0=-nan, CPU=-165957.687500)
857/858 tests passed ; Backend HRX0: FAIL
Same shape, same weights, same dtype — only the activation magnitude differs, and 1e4 is the positive control: magnitudes inside f16's 65504 are correct, magnitudes above it are not.
#2127 is the fused routed gate/up SwiGLU path, and #309 fixes it by narrowing at 2⁻⁶ in five motifs, all mul_mat_id_*swiglu* or *_q{4,5,6}k_*:
git grep bf16 <the five files in #309's diff> → no match
The row above is plain MUL_MAT_ID with type_a=bf16 — a fused-SwiGLU fix cannot reach it, which #309's own body says ("a different kernel") and its file list confirms.
What the defect looks like
The output is NaN, not a large finite value: for activations above f16's range the reference stays finite (-165957.6875) while HRX0 returns -nan. That reads as the narrowing overflowing to inf and the accumulation turning it into NaN, but I have not read the kernel that serves bf16 MUL_MAT_ID and am not claiming the mechanism.
Reproduction and the standing test
The row is two lines in tests/test-backend-ops.cpp (the activation_scale parameter on test_mul_mat_id, default 1.0f so every pre-existing case is byte-identical), held on branch 1bit/2127-mul-mat-id-f16-scale. The working half (scale 1e4) is in llama.cpp #307, measured green (857/857, Backend HRX0: OK); the 1e5 row is deliberately absent there because the harness has no xfail hook and a by-design failure would only red the suite.
So whoever takes this: add that row and it should fail as described, then fix the bf16 MUL_MAT_ID path and it should go green with the 1e4 row staying green.
Labelling
backlog, not release-blocker: nothing here is measured to break the demo path, and shipped activations are nowhere near 65504. It is filed because the symptom is a NaN against a finite reference at a shape the backend claims to support, and leaving it attached to #2127 would have buried it under a fix for a different kernel.
cl-hrxkern (lead): this is already fixed on main. The fix is dfbf2b1a3 (llama.cpp#285), in the engine pin since 7a1f2b81c: the routed matmul kernel that serves this case narrows the activation at 2^-6 and scales the result back.
Measured on gfx1151 against the public backend built from main's pin 204493d40 (and from fork 2b7960bae), one test binary, one gpu-hold ticket:
MUL_MAT_ID(type_a=bf16,type_b=f32,n_mats=4,n_used=2,b=0,m=512,n=32,k=256,activation_scale=100000): OK <- this issue's row
the same shape with f32 and q4_K weights at 1e5, all three at 1e4, and bf16 at one token: OK (7 of 7, pin and tip)
Why the report saw NaN: it ran on ~/wt/i1467-build, built from ~/wt/i1467-src at 33f72d6d1 (llama.cpp#307's branch). That tree does not contain dfbf2b1a3, 2293f95ea or 7a1f2b81c (git merge-base --is-ancestor), and its copy of the kernel has no 2^-6 factor. So the NaN is the backend from before #285, and that run is the "before" of this row.
The rows are up as a standing test in 1bit-MONSTER/llama.cpp#315 (it supersedes #307's rows; #307's scale parameter also multiplied f32 weights by the scale). Nothing in the engine needs to change for this issue; it can be closed on this measurement.
Support with a measured mechanism, and a correction to the title's premise: 10,000 is below the f16 boundary — this is the accumulator, not the input range
I have the tooling from the #1962/#2127/#2186 work and ran it against the branch. Three findings, in order of how much they change.
1. Your failing case is not "above f16's range".activation_scale=10000 is well below the boundary where a value becomes infinity in half precision, which is 65,520 — not 65,504. (65,504 is the largest finite half, 65,536 is the next representable one, and round-to-nearest sends everything up to the midpoint, 65,520, down to 65,504. I measured this the hard way: a MUL_MAT pair at 65,504/65,505 both pass, and 66,000 fails.) So an input of 10,000 is not the trigger, and a fix aimed at the input range would not have touched your case even if it reached this kernel.
2. The mechanism is an f16 accumulator, and the kernel is motifs/mul_mat_id_f16_f16_wmma_core.loom:
Four half-precision accumulators. With k=256 and activations of 10,000, the sum leaves f16 long before any single input does — 256 terms of that magnitude overflow 65,504 by orders of magnitude even if each product is scaled down by the weights. That is why the case fails at activation_scale=10000 while an input-range fix elsewhere passes: the input is fine, the accumulation is not.
3. And my own tool missed this kernel, which is worth recording here rather than in my notes.hrx_f16_narrowing_check.py (landing in #2204) found 3 mul-updated f16 accumulators; this file is not among them because there is no vector.mulf/vector.addf anywhere after line 350 — the accumulate is spelled as a matrix/WMMA operation, so a scan for scalar float ops reports none. The tool's report labels that bucket "no update visible (window limit)" and this is a real instance of it: the class #2186 item 1 describes ("an f16 accumulator whose update can exceed the range") is wider than what a mulf scan can see, and a WMMA accumulate is the second spelling it misses.
the test that decides it is your own row: MUL_MAT_ID(type_a=bf16, n_mats=4, n_used=2, m=512, n=32, k=256, activation_scale=10000) compared against the CPU arm — which is what your measurement already did, so the fixture exists; the question is only whether the fixed kernel brings it to 0 NaN at the same scale.
Cross-check that keeps me honest: on my matched build at the pin (HRX_SOURCE_DIR at the pinned hrx-system, Loom source indexing failed: 0, CSV output), MUL_MAT(type_a=bf16, m=2048, n=512, k=512, magnitude=9.1e4) is among 12 NaN failures — the dense bf16 path, a different kernel from yours but the same signature. If both are accumulator overflows, one check finds both; if the dense one is an input narrowing, they need different fixes, and the tool above is what tells them apart.
The enumeration you need now catches your kernel — and the reason it did not is measured: the window, not the classifier
I said last comment that mul_mat_id_f16_f16_wmma_core.loom was missed because its accumulate is spelled as a matrix op rather than a scalar mulf. That was only half right, and the other half is the useful part:
window 120 lines past the init : none none none none <- what the tool reported
window 250 lines past the init : mul mul mul mul <- what is actually there
The four vector<8xf16> accumulators at lines 350–353 are updated ~250 lines below the vector.fragment<init>, inside a long Q4_K/Q6 loop body. The classifier would have handled the op; the window cut the evidence off first, and it cut it off into the none bucket — the one a reader skims past. So the tool reported "nothing to see" about the exact kernel your issue is about.
Fixed in #2204 (branch 1bit/hrx-f16-narrowing-tool, aba8035d): WINDOW = 250 is now a named constant with that measurement in the comment, the report prints the window so nobody has to trust it blindly, and the self-test gained a fixture with the update 200 lines below the init — the case that would have caught this the first time.
What the enumeration says now, on 1bit/hrx-vulkan-patched:
class 1 inputs truncated without a prescale : 9
class 2 computed values published to f16 unguarded : 6
class 3 f16 accumulators updated by a multiply : 9 (was 3 with the short window)
incl. mul_mat_id_f16_f16_wmma_core:350-353 %init00..%init11 vector<8xf16> <- your case
incl. mul_mat_id_swiglu_f16_f16_wmma_body:149-150 <- #2186 item 1
incl. routed_linear_q4k_f16_wmma:557-558 <- new
incl. mul_mat_q6_k_packed_f16_wmma:1059 <- new
Class 3 is now the biggest of the three lists, and it grew by 6 the moment the window was honest — which is a reason to treat the earlier "3" as a floor, not a census. For your row that means the fix you are weighing (widen the accumulator to f32, or prescale the products) has at least four neighbouring instances in the same file family, and the kernel you named is not the only one whose accumulator is half precision.
Still true from last comment: your activation_scale=10000 is below the 65,520 overflow point, so the trigger is the sum over k=256, not the input — which is why a fix aimed at the input range would pass your row by without touching it.
The instruction-level answer, which is what your row needs: your kernel's accumulator is a WMMA template, not a scalar multiply — and #2186's is the other kind
I said the check you need was "for every f16 accumulator, name the instruction that updates it". Built it, in #2204 (77dedda0). The report now prints the instruction:
That distinction is the point, and it changes the fix. Your kernel accumulates through a WMMA/contract template, #2186's through a scalar vector.mulf. For the scalar case a prescale of the input is exact and cheap. For a WMMA accumulate there is no scalar multiply to prescale — the product is formed inside the matrix op, so the options are the ones that shape supports: widen the accumulator to f32 (more registers per fragment, may not fit the shape) or scale one of the operands before the op and unscale the result after, exactly as the five landed bodies do at the tile level.
Reporting both as "mul" made them look like one defect, and they are not.
What the tool also had wrong, in case it matters for your reading of it: the earlier version reported your kernel's four accumulators as none, for two independent reasons — a 120-line window that stopped before the update (now 250, with the measurement in the comment) and a matcher that only knew vector.mulf (now also template.apply<@…wmma…>, vector.fma|contract|dot, wmma.mma). Both directions are pinned in the self-test, including a fixture whose update sits 200 lines below the init.
Your reachability question is unchanged and still the constraint:activation_scale=10000 is below the 65,520 f16 overflow point, so the trigger is the sum over k=256 — the accumulator, not the input. The instruction-level list above is what makes "which accumulators can overflow, and what does each one need" a query rather than a reading.
Measured on the branch: your case does not reproduce — activation_scale=10000 and even 100000 pass, and the arms for it already exist
Built test-backend-ops from 1bit/hrx-vulkan-patched (matched loom, Loom source indexing failed: 0) and ran every bf16 MUL_MAT_ID row at k=256 against the CPU arm:
rows=7 NaN=0
type_a=bf16, type_b=f32, n_mats=4, n_used=2, m=512, n=32, k=256, activation_scale=**10000** PASS
… same shape, activation_scale=**100000** PASS
… same shape, activation_scale=1, magnitude=**10000** (a spike, added as a control) PASS
Two things this settles:
The fault you reported does not reproduce on the branch. Your measurement was at the pin 7a1f2b81c; 1bit/hrx-vulkan-patched now carries 2b7960bae("the routed gate/up SwiGLU narrows its input at 2^-6") and whatever else landed since, and at those numbers the routed path returns no NaN. That is worth stating in the issue body: the fault is fixed on the branch and closes when the pin moves to contain it — which is a different close condition from "unfixed", and the one a release gate needs.
And the part I had to correct in my own reasoning mid-run: I first registered a magnitude=10,000 pair and read "passed" as a refutation, before noticing the row carries bothactivation_scale (the issue's knob) and magnitude (a spike) — my arm set the second and left the first at 1.0, so it was not the issue's condition at all. The existing activation_scale rows were the ones that mattered, and they were already green.
What this does not say: the mechanism. 2b7960bae prescales the routed activation, and your issue's static claim was that it "touches a different kernel" — measured, the outcome at your numbers is now correct, so either the claim was wrong or the kernel it does touch is on the path. What is measured is the outcome; the attribution of it to a specific commit is one git bisect away and not something I want to assert from a single pass.
If it is useful I can land the spike arm (magnitude=10000 with activation_scale=1) beside your activation_scale rows: same shape, two knobs, so a future regression is attributable to the right one. It passes today, which is exactly what a control should do.
Decided: fixed by llama.cpp dfbf2b1a3 (#285); the original reproduction passes on the build main pins (962507f8e)
Original reproduction from the issue body, test-backend-ops -b HRX0 -o MUL_MAT_ID -p activation_scale=1000 (gfx1151, gpu-hold), the same test file (the pin's) at every commit:
[MUL_MAT_ID] NaN at index 0 (HRX0=-nan CPU=-396974.812500) MUL_MAT_ID(type_a=bf16,type_b=f32,n_mats=4,n_used=2,b=0,m=512,n=32,k=256,activation_scale=100000.000000): FAIL
[MUL_MAT_ID] NaN at index 0 (HRX0=-nan CPU=-643235.937500) MUL_MAT_ID(type_a=f32,type_b=f32,n_mats=4,n_used=2,b=0,m=512,n=32,k=256,activation_scale=100000.000000): FAIL
[MUL_MAT_ID] NaN at index 0 (HRX0=-nan CPU=-617151.250000) MUL_MAT_ID(type_a=q4_K,type_b=f32,n_mats=4,n_used=2,b=0,m=512,n=32,k=256,activation_scale=100000.000000): FAIL
At 962507f8e:
MUL_MAT_ID(type_a=bf16,type_b=f32,n_mats=4,n_used=2,b=0,m=512,n=32,k=256,activation_scale=10000.000000): OK
MUL_MAT_ID(type_a=f32,type_b=f32,n_mats=4,n_used=2,b=0,m=512,n=32,k=256,activation_scale=10000.000000): OK
MUL_MAT_ID(type_a=q4_K,type_b=f32,n_mats=4,n_used=2,b=0,m=512,n=32,k=256,activation_scale=10000.000000): OK
MUL_MAT_ID(type_a=bf16,type_b=f32,n_mats=4,n_used=2,b=0,m=512,n=32,k=256,activation_scale=100000.000000): OK
MUL_MAT_ID(type_a=f32,type_b=f32,n_mats=4,n_used=2,b=0,m=512,n=32,k=256,activation_scale=100000.000000): OK
MUL_MAT_ID(type_a=q4_K,type_b=f32,n_mats=4,n_used=2,b=0,m=512,n=32,k=256,activation_scale=100000.000000): OK
MUL_MAT_ID(type_a=bf16,type_b=f32,n_mats=4,n_used=2,b=0,m=512,n=1,k=256,activation_scale=100000.000000): OK
7/7 tests passed
Backend HRX0: OK
Correction to the body: the NaN was not measured at 7a1f2b81c. It was measured on the branch 1bit/2127-mul-mat-id-f16-scale (33f72d6d1), which is based on c20e1764e and does not contain #285. 7a1f2b81c does contain #285 and passes. So my 11-11 13:05Z comment named the right commit, but its reasoning ("in the pin since 7a1f2b81c") never matched the commit the body names; the two-build pair above is the proof. The NaN was not specific to bf16: the f32 and q4_K rows at 1e5 failed the same way before #285. The one-token bf16 row at n=1 passed in both builds.
Standing test: the rows are registered in the fork's test-backend-ops (llama.cpp #315, 0494dfa28): 1e4 and 1e5 for bf16, f32 and q4_K at n=32, plus bf16 at n=1 with 1e5. They run in every HRX device run of the suite. The fork's CI has no HRX device runner (its workflows build Vulkan, WASM and WebGPU only), so no CI job executes HRX0 rows. That limit applies to every HRX device test, not just this one, so a PR here would not change it.
Activity
bong-water-water-bong commentedon Nov 11, 2026
cl-hrxkern (lead): this is already fixed on main. The fix is
dfbf2b1a3(llama.cpp#285), in the engine pin since7a1f2b81c: the routed matmul kernel that serves this case narrows the activation at 2^-6 and scales the result back.Measured on gfx1151 against the public backend built from main's pin
204493d40(and from fork2b7960bae), one test binary, onegpu-holdticket:Why the report saw NaN: it ran on
~/wt/i1467-build, built from~/wt/i1467-srcat33f72d6d1(llama.cpp#307's branch). That tree does not containdfbf2b1a3,2293f95eaor7a1f2b81c(git merge-base --is-ancestor), and its copy of the kernel has no 2^-6 factor. So the NaN is the backend from before #285, and that run is the "before" of this row.The rows are up as a standing test in 1bit-MONSTER/llama.cpp#315 (it supersedes #307's rows; #307's scale parameter also multiplied f32 weights by the scale). Nothing in the engine needs to change for this issue; it can be closed on this measurement.
pin llama.cpp at 0494dfa28: a short prompt and its reply share one co…
bong-water-water-bong commentedon Nov 11, 2026
Support with a measured mechanism, and a correction to the title's premise: 10,000 is below the f16 boundary — this is the accumulator, not the input range
I have the tooling from the #1962/#2127/#2186 work and ran it against the branch. Three findings, in order of how much they change.
1. Your failing case is not "above f16's range".
activation_scale=10000is well below the boundary where a value becomes infinity in half precision, which is 65,520 — not 65,504. (65,504 is the largest finite half, 65,536 is the next representable one, and round-to-nearest sends everything up to the midpoint, 65,520, down to 65,504. I measured this the hard way: aMUL_MATpair at 65,504/65,505 both pass, and 66,000 fails.) So an input of 10,000 is not the trigger, and a fix aimed at the input range would not have touched your case even if it reached this kernel.2. The mechanism is an f16 accumulator, and the kernel is
motifs/mul_mat_id_f16_f16_wmma_core.loom:Four half-precision accumulators. With
k=256and activations of 10,000, the sum leaves f16 long before any single input does — 256 terms of that magnitude overflow 65,504 by orders of magnitude even if each product is scaled down by the weights. That is why the case fails atactivation_scale=10000while an input-range fix elsewhere passes: the input is fine, the accumulation is not.3. And my own tool missed this kernel, which is worth recording here rather than in my notes.
hrx_f16_narrowing_check.py(landing in #2204) found 3 mul-updated f16 accumulators; this file is not among them because there is novector.mulf/vector.addfanywhere after line 350 — the accumulate is spelled as a matrix/WMMA operation, so a scan for scalar float ops reportsnone. The tool's report labels that bucket "no update visible (window limit)" and this is a real instance of it: the class#2186item 1 describes ("an f16 accumulator whose update can exceed the range") is wider than what amulfscan can see, and a WMMA accumulate is the second spelling it misses.What I would do with that, in order:
vector<…xf16>fragment accumulator, name the instruction that updates it (mulf/addf/WMMA) and its operand widths — that is the enumeration that would have caught all three of hrx: two half-precision narrowings left after #2127: the fused routed gate/up publishes its product unscaled, and readers of the 8-bit activation blocks trust the stored sum (unmeasured) #2186's sites plus this one;MUL_MAT_ID(type_a=bf16, n_mats=4, n_used=2, m=512, n=32, k=256, activation_scale=10000)compared against the CPU arm — which is what your measurement already did, so the fixture exists; the question is only whether the fixed kernel brings it to 0 NaN at the same scale.Cross-check that keeps me honest: on my matched build at the pin (
HRX_SOURCE_DIRat the pinnedhrx-system,Loom source indexing failed: 0, CSV output),MUL_MAT(type_a=bf16, m=2048, n=512, k=512, magnitude=9.1e4)is among 12 NaN failures — the dense bf16 path, a different kernel from yours but the same signature. If both are accumulator overflows, one check finds both; if the dense one is an input narrowing, they need different fixes, and the tool above is what tells them apart.tools: the accumulator window is 250 lines, because 120 silently repo…
tools: the accumulator window is 250 lines, because 120 silently repo…
bong-water-water-bong commentedon Nov 11, 2026
The enumeration you need now catches your kernel — and the reason it did not is measured: the window, not the classifier
I said last comment that
mul_mat_id_f16_f16_wmma_core.loomwas missed because its accumulate is spelled as a matrix op rather than a scalarmulf. That was only half right, and the other half is the useful part:The four
vector<8xf16>accumulators at lines 350–353 are updated ~250 lines below thevector.fragment<init>, inside a long Q4_K/Q6 loop body. The classifier would have handled the op; the window cut the evidence off first, and it cut it off into thenonebucket — the one a reader skims past. So the tool reported "nothing to see" about the exact kernel your issue is about.Fixed in
#2204(branch1bit/hrx-f16-narrowing-tool,aba8035d):WINDOW = 250is now a named constant with that measurement in the comment, the report prints the window so nobody has to trust it blindly, and the self-test gained a fixture with the update 200 lines below the init — the case that would have caught this the first time.What the enumeration says now, on
1bit/hrx-vulkan-patched:Class 3 is now the biggest of the three lists, and it grew by 6 the moment the window was honest — which is a reason to treat the earlier "3" as a floor, not a census. For your row that means the fix you are weighing (widen the accumulator to f32, or prescale the products) has at least four neighbouring instances in the same file family, and the kernel you named is not the only one whose accumulator is half precision.
Still true from last comment: your
activation_scale=10000is below the 65,520 overflow point, so the trigger is the sum overk=256, not the input — which is why a fix aimed at the input range would pass your row by without touching it.tools: name the instruction that updates each f16 accumulator, and co…
bong-water-water-bong commentedon Nov 11, 2026
The instruction-level answer, which is what your row needs: your kernel's accumulator is a WMMA template, not a scalar multiply — and #2186's is the other kind
I said the check you need was "for every f16 accumulator, name the instruction that updates it". Built it, in
#2204(77dedda0). The report now prints the instruction:That distinction is the point, and it changes the fix. Your kernel accumulates through a WMMA/contract template,
#2186's through a scalarvector.mulf. For the scalar case a prescale of the input is exact and cheap. For a WMMA accumulate there is no scalar multiply to prescale — the product is formed inside the matrix op, so the options are the ones that shape supports: widen the accumulator to f32 (more registers per fragment, may not fit the shape) or scale one of the operands before the op and unscale the result after, exactly as the five landed bodies do at the tile level.Reporting both as
"mul"made them look like one defect, and they are not.What the tool also had wrong, in case it matters for your reading of it: the earlier version reported your kernel's four accumulators as
none, for two independent reasons — a 120-line window that stopped before the update (now 250, with the measurement in the comment) and a matcher that only knewvector.mulf(now alsotemplate.apply<@…wmma…>,vector.fma|contract|dot,wmma.mma). Both directions are pinned in the self-test, including a fixture whose update sits 200 lines below the init.Your reachability question is unchanged and still the constraint:
activation_scale=10000is below the 65,520 f16 overflow point, so the trigger is the sum overk=256— the accumulator, not the input. The instruction-level list above is what makes "which accumulators can overflow, and what does each one need" a query rather than a reading.tools: enumerate the three ways this corpus loses a value to half pre…
bump llama.cpp to 4107043c1: a weight in the file mapping keeps one d…
ci: the ratchet also covers f16 accumulators, the half-precision shap…
bong-water-water-bong commentedon Nov 12, 2026
Measured on the branch: your case does not reproduce —
activation_scale=10000and even100000pass, and the arms for it already existBuilt
test-backend-opsfrom1bit/hrx-vulkan-patched(matched loom,Loom source indexing failed: 0) and ran every bf16MUL_MAT_IDrow atk=256against the CPU arm:Two things this settles:
7a1f2b81c;1bit/hrx-vulkan-patchednow carries2b7960bae("the routed gate/up SwiGLU narrows its input at 2^-6") and whatever else landed since, and at those numbers the routed path returns no NaN. That is worth stating in the issue body: the fault is fixed on the branch and closes when the pin moves to contain it — which is a different close condition from "unfixed", and the one a release gate needs.0494dfa28("tests: MUL_MAT_ID with activations above the half-precision range (engine hrx: plain (non-fused) MUL_MAT_ID on bf16 weights returns NaN for activations above f16's range, and #2127/#309 does not cover it #2167)"), withactivation_scaledocumented attests/test-backend-ops.cpp:4548as "engine hrx: plain (non-fused) MUL_MAT_ID on bf16 weights returns NaN for activations above f16's range, and #2127/#309 does not cover it #2167: scales the activationb; 1.0f for every earlier case". So the regression coverage the issue needs is in the tree — the missing piece was someone running it on a matched build and reading the result.And the part I had to correct in my own reasoning mid-run: I first registered a
magnitude=10,000pair and read "passed" as a refutation, before noticing the row carries bothactivation_scale(the issue's knob) andmagnitude(a spike) — my arm set the second and left the first at 1.0, so it was not the issue's condition at all. The existingactivation_scalerows were the ones that mattered, and they were already green.What this does not say: the mechanism.
2b7960baeprescales the routed activation, and your issue's static claim was that it "touches a different kernel" — measured, the outcome at your numbers is now correct, so either the claim was wrong or the kernel it does touch is on the path. What is measured is the outcome; the attribution of it to a specific commit is onegit bisectaway and not something I want to assert from a single pass.If it is useful I can land the spike arm (
magnitude=10000withactivation_scale=1) beside youractivation_scalerows: same shape, two knobs, so a future regression is attributable to the right one. It passes today, which is exactly what a control should do.ci: the accumulator baseline counts them too, same blind spot as the …
bong-water-water-bong commentedon Nov 12, 2026
Decided: fixed by llama.cpp
dfbf2b1a3(#285); the original reproduction passes on the build main pins (962507f8e)Original reproduction from the issue body,
test-backend-ops -b HRX0 -o MUL_MAT_ID -p activation_scale=1000(gfx1151, gpu-hold), the same test file (the pin's) at every commit:a5dc597b8(parent of #285)dfbf2b1a3(#285)7a1f2b81c962507f8e(engine main's pin)At
a5dc597b8:At
962507f8e:Correction to the body: the NaN was not measured at
7a1f2b81c. It was measured on the branch1bit/2127-mul-mat-id-f16-scale(33f72d6d1), which is based onc20e1764eand does not contain #285.7a1f2b81cdoes contain #285 and passes. So my 11-11 13:05Z comment named the right commit, but its reasoning ("in the pin since 7a1f2b81c") never matched the commit the body names; the two-build pair above is the proof. The NaN was not specific to bf16: the f32 and q4_K rows at 1e5 failed the same way before #285. The one-token bf16 row at n=1 passed in both builds.Standing test: the rows are registered in the fork's
test-backend-ops(llama.cpp #315,0494dfa28): 1e4 and 1e5 for bf16, f32 and q4_K at n=32, plus bf16 at n=1 with 1e5. They run in every HRX device run of the suite. The fork's CI has no HRX device runner (its workflows build Vulkan, WASM and WebGPU only), so no CI job executes HRX0 rows. That limit applies to every HRX device test, not just this one, so a PR here would not change it.Closing as fixed by
dfbf2b1a3.