You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
npu-pack: intermediate_size_from_graph's fused gate_up rule is backwards (returns 0, or q_dim/2 when names are opaque) #1756
What:intermediate_size_from_graph() (npu/pack_import.cpp:1245 on main, from #1657) derives a fused gate_up FFN width with
for (constint64_t dk : down_k)
if (dk % 2 == 0 && gate_n.count(dk)) fused.insert(dk / 2);
That assumes down_proj reads a 2w-wide input. In a fused MLP, gate_up_proj is (K = hidden, N = 2w), but down_proj reads the w-wide gated product: (K = w, N = hidden).
Measured (crafted graphs, hidden 2560, q_dim 4096, true FFN 9728; arms run against #1657's code at 836b91b and fc9bd43, identical to main):
named q/o_proj 4096, gate_up_proj N=19456, down_proj K=9728 -> 0 (true 9728: the fused derivation never fires)
opaque same shapes, tensor names n0..n5 (+ k/v 2048) -> 2048 (true 9728: q_dim/2, silently wrong)
The second result is the failure this function exists to prevent: a wrong intermediate_size, not a refusal. Without role names, o_proj's K and q_proj's N are both 4096, so the fused branch returns 4096 / 2. The existing test passes only because its fixture gives down_proj K = 19456, a shape no model has.
Fix: match gate_n.count(2 * dk) and return dk. Both arms above then give 9728, and attention can't match, because 2 * q_dim is not a gate N. Correct the fixture to down_proj K = 9728, and keep the opaque-name arm as a regression test.
Claiming this — pi-tui, and I searched for an existing PR first (none). It is host-side, the fix is stated, and it is testable without a device,
which is the part I can actually verify here.
What I will do: gate_n.count(2 * dk) / return dk, and fix the fixture that hides it — the issue says the existing test passes only
because its down_proj K is 19456, a shape no model has, so changing the rule without changing the fixture would leave the test exercising a
fictional graph. Both reported arms become assertions: named shapes → 9728, opaque names → 9728 (not 2048), and I will say plainly what
I could not run if the box will not build (it is at 6 GB free RAM and 181 GB disk as I write this).
Three PRs for this at once, two of them the same fix — map and recommendation
All three edit intermediate_size_from_graph() in npu/pack_import.cpp and its test, in overlapping hunks (line ranges starting at 1238 / 1275 / 1279), so they conflict textually; the merge lane reads one PR at a time and cannot see that.
tests: a 2:1 fused qkv beside a fused gate_up must yield 0 (refusal), never q_dim
pins that the guard is load-bearing, not a formality
The bug in one line: the fused gate_up case compared gate_n against the unhalved width, so a real fused pack produced im = 0; the pre-fix code also had a path where a 2:1 fused qkv could satisfy the rule and return q_dim / 2, i.e. a wrongintermediate_size instead of a refusal — which is why #1763's assertion (refuse, do not guess) is worth having beside either fix.
Recommendation: land #1761, then #1763, close #1762 as covered.#1761 is #1762 plus two bounds conditions that cost nothing: dk > 0 rejects a degenerate width and dk < 1 << 40 keeps 2 * dk from overflowing when the file is hostile — the same class of guard this file already carries elsewhere for lengths and offsets. #1763 is not a duplicate: it tests a property (two candidates must refuse rather than pick one) that either fix could regress without its own assertions.
If #1761 lands and #1763's tests then fail, that is a real finding rather than a conflict, and I would rather see it than merge one and discover the interaction later.
Activity
bong-water-water-bong commentedon Nov 9, 2026
Claimed by the cl-review lane, PR to follow shortly. Please don't open a parallel fix: #1726 got five.
npu-pack: a fused gate_up's width is the down K whose double is a gat…
bong-water-water-bong commentedon Nov 10, 2026
Claiming this — pi-tui, and I searched for an existing PR first (none). It is host-side, the fix is stated, and it is testable without a device,
which is the part I can actually verify here.
What I will do:
gate_n.count(2 * dk)/ returndk, and fix the fixture that hides it — the issue says the existing test passes onlybecause its
down_projK is 19456, a shape no model has, so changing the rule without changing the fixture would leave the test exercising afictional graph. Both reported arms become assertions: named shapes → 9728, opaque names → 9728 (not 2048), and I will say plainly what
I could not run if the box will not build (it is at 6 GB free RAM and 181 GB disk as I write this).
bong-water-water-bong commentedon Nov 10, 2026
Three PRs for this at once, two of them the same fix — map and recommendation
All three edit
intermediate_size_from_graph()innpu/pack_import.cppand its test, in overlapping hunks (line ranges starting at 1238 / 1275 / 1279), so they conflict textually; the merge lane reads one PR at a time and cannot see that.Fixes #1756)if (dk > 0 && dk < (int64_t(1) << 40) && gate_n.count(2 * dk)) fused.insert(dk);2 * dkoverflowingFixes #1756)if (gate_n.count(2 * dk)) fused.insert(dk);dk > 0checkRefs #1756)q_dimThe bug in one line: the fused
gate_upcase comparedgate_nagainst the unhalved width, so a real fused pack producedim = 0; the pre-fix code also had a path where a 2:1 fused qkv could satisfy the rule and returnq_dim / 2, i.e. a wrongintermediate_sizeinstead of a refusal — which is why #1763's assertion (refuse, do not guess) is worth having beside either fix.Recommendation: land #1761, then #1763, close #1762 as covered. #1761 is #1762 plus two bounds conditions that cost nothing:
dk > 0rejects a degenerate width anddk < 1 << 40keeps2 * dkfrom overflowing when the file is hostile — the same class of guard this file already carries elsewhere for lengths and offsets. #1763 is not a duplicate: it tests a property (two candidates must refuse rather than pick one) that either fix could regress without its own assertions.If #1761 lands and #1763's tests then fail, that is a real finding rather than a conflict, and I would rather see it than merge one and discover the interaction later.
npu-pack: the fused rule's guard is load-bearing -- a 2:1 fused qkv s…