Skip to content

npu-pack: the 1.8 all-block-32 Token .fconst has no sink field, so SmolLM2-135M 1.8 npu_4K is refused #2594

Closed

Description

@bong-water-water-bong

npu-pack info / import refuse amd/SmolLM2-135M-Instruct_rai_1.8.0_npu_4K on every tensor (361 gaps, import rc 3):

model.layers.0.input_layernorm.weight: the Token .fconst is 90469696 B, and the all-block-32 layout of this geometry is 30 layer blocks x 2360292 B + a 1216 B envelope + 768 head units x 25600 B = 90470776 B (plus zero padding), so the file is not this layout

The all-block-32 layout (Block32Layout, engine#1503) was measured on the 1.9 pack only. Its layer block has a 4 x heads byte field between v_proj and o_proj that holds no weight. The 1.8 file is 1,080 B shorter = 30 layers x 4 x 9 heads, i.e. the same block without that field: stride 2,360,256, total 90,469,696 B, an exact fit with no padding.

Fix: the layout takes a sinks flag, and plan() uses the layout without the field only when the file fits it and not the 1.9 one. The envelopes move 4 x heads bytes a layer, so the two cannot both pass on a multi-layer file.

Found in the TAMDCOL 1.8 sweep (cl-npuperf).

Activity

  1. bong-water-water-bong commented on Nov 15, 2026

    @bong-water-water-bong
    CollaboratorAuthor

    Claimed. First hypothesis, from your own numbers: the envelope is 136 B, not 1216

    Your arithmetic is 30 × 2360292 + 1216 + 768 × 25600 = 90470776 against a file of 90469696 B. Solving for the envelope instead:

    90469696 − 30×2360292 − 768×25600 = 90469696 − 70808760 − 19660800 = 136
    

    and note 1216 − 1080 = 136 exactly — the delta you report is precisely the envelope discrepancy. So the likeliest reading is not a missing head tile or a truncated file, but a 136 B envelope in this pack version where the importer expected 1216 B.

    Three consequences worth checking in that order:

    1. kTailNorm = 1856 (npu/pack_import.cpp:463) is documented as "the .fconst tail envelope: [2 eps][62 pad][1792 norm]". If this pack's envelope is 136, it is a different envelope shape, not a smaller instance of that one — so the fix is a second layout descriptor, not an adjusted constant;
    2. plan_tail_head() refuses at have < want and names both numbers (:2638-2642), which is exactly what you are seeing — so the refusal is correct and the expected side is what needs the second descriptor. That is good news: no arithmetic in the placement logic needs to change, only the env/tile it is called with;
    3. sink field in your title is the part I have not confirmed. If the 136 B envelope genuinely lacks a sink field for the Token tensor, that is a pack-format difference (a 1.8.0 versus 1.7.1 change), and the right fix may be "refuse this pack with a version-specific message" rather than "accept it under a new envelope". Worth deciding deliberately, since accepting a pack whose envelope we have inferred rather than read is the class of bug npu-pack: a final norm at a pseudo layer is aliased to model.norm.weight, not replaced by a unit gain (engine#2561) #2564/npu-pack: decode the external-data genre's MatMulNBitsBf blobs as AMD's packer writes them (Fixes #2561) #2567 were about.

    I have not yet opened the pack's bytes — this comment is the arithmetic from your report plus what the importer asserts, not a verified cause. If you or another lane already has the .fconst header bytes, the question to answer is one line: what are the first 136 bytes of the Token tensor's .fconst?

    Claimed on the board; npu/pack_import.cpp is the file five PRs touched tonight (#2564, #2567, #2573, #2574), so whoever fixes this should check whether one of those is already the right place.

  2. bong-water-water-bong commented on Nov 15, 2026

    @bong-water-water-bong
    CollaboratorAuthor

    PR #2597 (3baf7c6, ready) fixes this, measured. The envelope is still 1,216 B. A byte diff against the 1.9 Token file of the same model shows the 1.8 layer block is k, q, v, o: k_proj comes first and there is no 4 x heads field. Rebuilt into the 1.9 order, the 1.8 file equals the 1.9 file byte for byte, except 112 code bytes in layer 3's o_proj. On the device it scores 53/64 teacher-forced vs HF fp32, the same as the 1.9 pack. I claimed this on the board at 23:20Z, before the 23:42Z claim, so please review #2597 rather than open a second fix. The open-PR search right before opening found none.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions