You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
npu-pack: the 1.7 -onnx-ryzenai-npu Token file of a tied model (Phi-4-mini) has no head tiles #2658
Phi-4-mini-instruct-onnx-ryzenai-npu (Ryzen AI 1.7) is refused with 320 gaps, "fewer than 6 RMSNorm epsilon headers in the first GiB". That happens even with #2651, which fixes the other 1.7 npu refusals.
Measured from its own size: the Token file dd_metastate_Llm_Token_MatMulNBits_2_0.fconst is 1,758,724,096 B, which is exactly 32 x 54,960,128 B layer strides of the 1.7 headerless form (#2638: hidden 3072, IM 8192, 24/8 heads, head_dim 128). Nothing follows the layers. The model ties lm_head to the embedding (vocab 200,064), so the export leaves out the 3,126 head tiles that every other 1.7 npu pack carries.
The 1.7 rewrite needs the head in the tile format. So the fix has to build it from the pack's embedding (or reuse it in whatever form the forward takes for a tied head), with a byte-exactness check against an independent decode. The 1.8 npu_4K twin's Token tail should show the target layout. Found in the TAMDCOL 1.7 sweep (cl-npuperf).
I verified the arithmetic in this issue exactly, from #2651's own constants, and it names why the refusal happens — and why it is misleading.
The numbers match to the byte
Re-deriving the 1.7 headerless stride from fconst17_geometry()'s arithmetic (the one #2651 introduces), with this model's dims — hidden 3072, IM 8192, 24 heads / 8 kv, head_dim 128, vocab 200,064, 32 layers:
tile_h = tile_o = 108,800 n_q = 48 n_kv = 16 n_o = 48
stride = 54,960,128 <- this issue says 54,960,128 MATCH
32 x stride = 1,758,724,096 <- this issue says 1,758,724,096 MATCH
head = 3,126 x 108,800 = 340,108,800 B
So the file is exactly layers × stride, with nothing after — and the 340,108,800 B that a sibling pack would carry is precisely the 3,126 head tiles this one omits. Independent confirmation of the issue's central claim, from constants rather than from the file.
That tells us where the refusal actually comes from
fconst17_geometry() computes
want = layers x stride + head
and refuses when size != want (modulo the whole-tile padding it tolerates after the head). For a tied pack, size == layers × stride — so the geometry path rejects a stream whose stride it has computed correctly, and the caller falls through to the epsilon-header vote, which is what emits "fewer than 6 RMSNorm epsilon headers in the first GiB".
The message names the wrong cause. There is nothing wrong with the epsilon headers; the arithmetic declined because a head is missing, and the fallback then blamed an anchor it could not find. A reader with this pack is sent to the epsilon machinery — which is what #2638/#2651 are about — when the actual reason is "this export has no head".
The change I would make, in the order I would make it
fconst17_geometry() should accept size == layers × stride as a named variant — head absent because the model ties lm_head to the embedding — and set g->head = 0 plus a flag saying so. That places this pack without inventing bytes, and it is the same predicate shape the function already uses: it knows the stride, so it can tell "short by exactly a head" from "the wrong geometry entirely".
The refusal should then name it: "the pack carries no head tiles (size == layers x stride; the model ties lm_head to the embedding)", not the epsilon message.
Building the head from the embedding (the issue's own fix) is the real work and follows from (1): the geometry is placeable, one thing is missing, and the forward's tied-head path already exists at the container level (engine#1503's selfcheck_tied). Whether the 1.7 rewrite should synthesise 3,126 tiles of 108,800 B or the reader should reuse the embedding in place is a decision for whoever owns that path — but it can only be made once the stream is located, which is what (1) buys.
Verified without the pack
The pack is not on this box (~/models has only the amd-rai/sweep logs), so I cannot reproduce the refusal end to end. Everything above is arithmetic against the constants in #2651 — which is why I am posting it as a finding and not as a patch: step 1 is testable with a synthetic stream (a layers × stride file with no head), the same way test_fconst17_known_geometry builds one, but I would want the refusal reproduced before writing the head-synthesis half.
Activity
bong-water-water-bong commentedon Nov 16, 2026
I verified the arithmetic in this issue exactly, from
#2651's own constants, and it names why the refusal happens — and why it is misleading.The numbers match to the byte
Re-deriving the 1.7 headerless stride from
fconst17_geometry()'s arithmetic (the one #2651 introduces), with this model's dims — hidden 3072, IM 8192, 24 heads / 8 kv, head_dim 128, vocab 200,064, 32 layers:So the file is exactly
layers × stride, with nothing after — and the 340,108,800 B that a sibling pack would carry is precisely the 3,126 head tiles this one omits. Independent confirmation of the issue's central claim, from constants rather than from the file.That tells us where the refusal actually comes from
fconst17_geometry()computesand refuses when
size != want(modulo the whole-tile padding it tolerates after the head). For a tied pack,size == layers × stride— so the geometry path rejects a stream whose stride it has computed correctly, and the caller falls through to the epsilon-header vote, which is what emits "fewer than 6 RMSNorm epsilon headers in the first GiB".The message names the wrong cause. There is nothing wrong with the epsilon headers; the arithmetic declined because a head is missing, and the fallback then blamed an anchor it could not find. A reader with this pack is sent to the epsilon machinery — which is what #2638/#2651 are about — when the actual reason is "this export has no head".
The change I would make, in the order I would make it
fconst17_geometry()should acceptsize == layers × strideas a named variant —headabsent because the model tieslm_headto the embedding — and setg->head = 0plus a flag saying so. That places this pack without inventing bytes, and it is the same predicate shape the function already uses: it knows the stride, so it can tell "short by exactly a head" from "the wrong geometry entirely"."the pack carries no head tiles (size == layers x stride; the model ties lm_head to the embedding)", not the epsilon message.selfcheck_tied). Whether the 1.7 rewrite should synthesise 3,126 tiles of 108,800 B or the reader should reuse the embedding in place is a decision for whoever owns that path — but it can only be made once the stream is located, which is what (1) buys.Verified without the pack
The pack is not on this box (
~/modelshas only theamd-rai/sweeplogs), so I cannot reproduce the refusal end to end. Everything above is arithmetic against the constants in #2651 — which is why I am posting it as a finding and not as a patch: step 1 is testable with a synthetic stream (alayers × stridefile with no head), the same waytest_fconst17_known_geometrybuilds one, but I would want the refusal reproduced before writing the head-synthesis half.