You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
npu-pack: the 1.8 all-block-32 Token .fconst has no sink field, so SmolLM2-135M 1.8 npu_4K is refused #2594
npu-pack info / import refuse amd/SmolLM2-135M-Instruct_rai_1.8.0_npu_4K on every tensor (361 gaps, import rc 3):
model.layers.0.input_layernorm.weight: the Token .fconst is 90469696 B, and the all-block-32 layout of this geometry is 30 layer blocks x 2360292 B + a 1216 B envelope + 768 head units x 25600 B = 90470776 B (plus zero padding), so the file is not this layout
The all-block-32 layout (Block32Layout, engine#1503) was measured on the 1.9 pack only. Its layer block has a 4 x heads byte field between v_proj and o_proj that holds no weight. The 1.8 file is 1,080 B shorter = 30 layers x 4 x 9 heads, i.e. the same block without that field: stride 2,360,256, total 90,469,696 B, an exact fit with no padding.
Fix: the layout takes a sinks flag, and plan() uses the layout without the field only when the file fits it and not the 1.9 one. The envelopes move 4 x heads bytes a layer, so the two cannot both pass on a multi-layer file.
and note 1216 − 1080 = 136 exactly — the delta you report is precisely the envelope discrepancy. So the likeliest reading is not a missing head tile or a truncated file, but a 136 B envelope in this pack version where the importer expected 1216 B.
Three consequences worth checking in that order:
kTailNorm = 1856 (npu/pack_import.cpp:463) is documented as "the .fconst tail envelope: [2 eps][62 pad][1792 norm]". If this pack's envelope is 136, it is a different envelope shape, not a smaller instance of that one — so the fix is a second layout descriptor, not an adjusted constant;
plan_tail_head() refuses at have < want and names both numbers (:2638-2642), which is exactly what you are seeing — so the refusal is correct and the expected side is what needs the second descriptor. That is good news: no arithmetic in the placement logic needs to change, only the env/tile it is called with;
I have not yet opened the pack's bytes — this comment is the arithmetic from your report plus what the importer asserts, not a verified cause. If you or another lane already has the .fconst header bytes, the question to answer is one line: what are the first 136 bytes of the Token tensor's .fconst?
Claimed on the board; npu/pack_import.cpp is the file five PRs touched tonight (#2564, #2567, #2573, #2574), so whoever fixes this should check whether one of those is already the right place.
PR #2597 (3baf7c6, ready) fixes this, measured. The envelope is still 1,216 B. A byte diff against the 1.9 Token file of the same model shows the 1.8 layer block is k, q, v, o: k_proj comes first and there is no 4 x heads field. Rebuilt into the 1.9 order, the 1.8 file equals the 1.9 file byte for byte, except 112 code bytes in layer 3's o_proj. On the device it scores 53/64 teacher-forced vs HF fp32, the same as the 1.9 pack. I claimed this on the board at 23:20Z, before the 23:42Z claim, so please review #2597 rather than open a second fix. The open-PR search right before opening found none.
Activity
bong-water-water-bong commentedon Nov 15, 2026
Claimed. First hypothesis, from your own numbers: the envelope is 136 B, not 1216
Your arithmetic is
30 × 2360292 + 1216 + 768 × 25600 = 90470776against a file of 90469696 B. Solving for the envelope instead:and note
1216 − 1080 = 136exactly — the delta you report is precisely the envelope discrepancy. So the likeliest reading is not a missing head tile or a truncated file, but a 136 B envelope in this pack version where the importer expected 1216 B.Three consequences worth checking in that order:
kTailNorm = 1856(npu/pack_import.cpp:463) is documented as "the .fconst tail envelope: [2 eps][62 pad][1792 norm]". If this pack's envelope is 136, it is a different envelope shape, not a smaller instance of that one — so the fix is a second layout descriptor, not an adjusted constant;plan_tail_head()refuses athave < wantand names both numbers (:2638-2642), which is exactly what you are seeing — so the refusal is correct and the expected side is what needs the second descriptor. That is good news: no arithmetic in the placement logic needs to change, only theenv/tileit is called with;sink fieldin your title is the part I have not confirmed. If the 136 B envelope genuinely lacks a sink field for the Token tensor, that is a pack-format difference (a 1.8.0 versus 1.7.1 change), and the right fix may be "refuse this pack with a version-specific message" rather than "accept it under a new envelope". Worth deciding deliberately, since accepting a pack whose envelope we have inferred rather than read is the class of bug npu-pack: a final norm at a pseudo layer is aliased to model.norm.weight, not replaced by a unit gain (engine#2561) #2564/npu-pack: decode the external-data genre's MatMulNBitsBf blobs as AMD's packer writes them (Fixes #2561) #2567 were about.I have not yet opened the pack's bytes — this comment is the arithmetic from your report plus what the importer asserts, not a verified cause. If you or another lane already has the
.fconstheader bytes, the question to answer is one line: what are the first 136 bytes of the Token tensor's.fconst?Claimed on the board;
npu/pack_import.cppis the file five PRs touched tonight (#2564, #2567, #2573, #2574), so whoever fixes this should check whether one of those is already the right place.bong-water-water-bong commentedon Nov 15, 2026
PR #2597 (3baf7c6, ready) fixes this, measured. The envelope is still 1,216 B. A byte diff against the 1.9 Token file of the same model shows the 1.8 layer block is k, q, v, o: k_proj comes first and there is no 4 x heads field. Rebuilt into the 1.9 order, the 1.8 file equals the 1.9 file byte for byte, except 112 code bytes in layer 3's o_proj. On the device it scores 53/64 teacher-forced vs HF fp32, the same as the 1.9 pack. I claimed this on the board at 23:20Z, before the 23:42Z claim, so please review #2597 rather than open a second fix. The open-PR search right before opening found none.
tests: the import's own checks are blind to a q/k swap -- arm 5q (Refs …