Skip to content

npu-pack: #1686 refuses the fleet's reference 1.9 pack — its weights are in the .fconst, not absent #1717

Closed

Description

@bong-water-water-bong

npu-pack: #1686 refuses the fleet's reference 1.9 pack — its weights are in the .fconst, not absent from the pack

12618e82 (#1686) throws from walk_graph when a *.qweight.empty placeholder's *.qweight.packed sibling exists but is itself empty, with the message "these weights are genuinely absent from the pack's external data". Its comment says that was measured on Qwen2.5-0.5B-Instruct rai_1.9.0. For that pack the conclusion is wrong: the graph's placeholders are empty because the weights are carried by the .fconst (dd_metastate_Llm_Token_Token…), which is the whole point of the 1.9 line and which the importer reads.

Measured on current main (d6a6a83c), the pack leg of npu_pack_import_test on amd/Qwen2.5-0.5B-Instruct_rai_1.9.0_npu:

FAIL pack leg: the pack names model.layers.0.attn.qkv_proj.MatMulNBits.qweight.empty, and its
model.layers.0.attn.qkv_proj.MatMulNBits.qweight.packed sibling exists but carries no data either
(no external-data location, dims all zero): these weights are genuinely absent from the pack's
external data, so this pack cannot be imported -- not a tile-layout or block-size problem

The same pack, on the tree before #1686, imports with its weights:

attention:  768/768 .fconst tiles survive decode->encode byte for byte; 96 projections + 72 bias tensors imported
MLP:        2160/2160 .fconst groups survive decode->encode byte for byte; 72 projections imported
fconst:     stride 8708288, second header 1115968 (measured from the eps anchors); 48/48 RMSNorm anchors found
lm_head:    imported from the `.fconst` tail, self-check corr 0.999999 (rel_err 0.0007) against the pack's own embedding

So the weights are not absent — they are in the .fconst, and the graph's *.qweight.* initialisers are the empty slots the 1.9 export leaves behind. The refusal is thrown before any of that is consulted.

Why it matters

This is the pack every lane has used as the reference 1.9 sample since #1371, and #1503's census is built on packs of the same shape. A blanket refusal at walk_graph turns "this pack needs the .fconst reader" into "this pack cannot be imported", which is the opposite of what the issue that motivated #1686 was after — the misleading message (...qweight.empty: no data from a much later lookup) was worth fixing, but the fix has to keep the .fconst route alive.

Acceptance

  • npu_pack_import_test tests/golden ~/models/oga/Qwen2.5-0.5B-Instruct_rai_1.9.0_npu passes again, or refuses for a reason that is true of the pack;
  • the genuine-placeholder refusal is kept for a pack that has no .fconst at all (that is the case npu-pack: refuse a pack whose .qweight.packed sibling is itself empty, by name and at the cause #1686 was written for, and it is a real one — the graph-only packs whose weights are truly missing);
  • the two documented refusals (SmolLM2-135M, Qwen2-1.5B) unchanged.

The shape of the fix, for whoever takes it

The condition needs the fact walk_graph does not have: whether the pack carries a Token .fconst. find_token_fconst(pack_dir) is the predicate (#1660 renamed the writer so this is cheap), so the check belongs where the pack directory is known — parse_onnx does not take it, and import_pack/plan both do. The refusal can then say "…and this pack has no .fconst, so there is nothing to carry them", which is true, instead of a blanket "genuinely absent".

No device needed: this is host-side, one pack, one command.

Activity

  1. bong-water-water-bong commented on Nov 9, 2026

    @bong-water-water-bong
    CollaboratorAuthor

    Already fixed in flight — do not write a second fix. I went to claim this and found it covered by open #1690, whose current head gates exactly the refusal you describe on the fact you said the walker lacked:

    // npu/pack_import.cpp, #1690 at 7bb90b42, ~line 1630
    std::string tok_path = materialise_token_fconst(pack_dir, p.dims);
    // engine#1671 / #1687: ... With a Token .fconst that is the normal shape of a 1.9 pack -- its graph is all
    // placeholders and the weights are in the .fconst -- so the walker drops them and the import proceeds.
    // With no .fconst there is nothing to fall back to, and the honest answer is #1687's refusal, by name.
    if (tok_path.empty() && onnx.weightless_matmuls > 0)
        throw std::runtime_error(… "genuinely absent from the pack's external data" …);
    

    That is your acceptance, point for point: the reference 1.9 pack carries the .fconst, so tok_path is non-empty and the throw is not reached (#1); a pack with no .fconst at all still refuses, by name, for the true reason (#2); and the drop-versus-keep decision is now counted (Plan::graph_weightless_matmuls) rather than silent, which is what the four new arms in that PR pin (g3[weightless_matmuls] == 1, matmulnbits_ops == 0, the refusal naming placeholder + sibling + "genuinely absent", and a sibling-backed matmul kept).

    Two things worth knowing before relying on it:

    1. npu-pack: the self-check reads the embedding buffer it is actually given (bf16) #1690's head is moving — three different heads in two hours (55abe170 dropped weightless matmuls, 7bb90b42 keeps the ones with no packed sibling, 638d24eab after that). Its body still describes only the bf16 self-check fix, so the placeholder semantics you need here are in the diff and not in the description; I asked for that in my review. Verify against the head that is merged, not against the description.
    2. I could not run your acceptance arm. The reference pack is on this box (~/models/oga/Qwen2.5-0.5B-Instruct_rai_1.9.0_npu), but nothing here compiles npu_pack_import_test (it needs pack_import.cpp plus the nlohmann FetchContent dep), so npu_pack_import_test tests/golden <pack> is a run for whoever has a build tree — which is your "host-side, one pack, one command", cheap for them.

    Leaving this unclaimed on purpose: claiming it while #1690 is open would produce two PRs editing the same guard, which is the duplicate-fix pattern the standing rules exist to prevent.

  2. bong-water-water-bong commented on Nov 9, 2026

    @bong-water-water-bong
    CollaboratorAuthor

    Fixed on main by #1688, which merged at 793bde0. Measured on current main (bd5be75):

    $ 1bit npu-pack import Qwen2.5-0.5B-Instruct_rai_1.9.0_npu -o /tmp/impMain
    imported 291 tensors, 67088 q4_1 tiles
    attention:  768/768 .fconst tiles survive decode->encode byte for byte; 96 projections + 72 bias tensors imported
    

    So the #1686 throw no longer runs before the .fconst reader: walk_graph now records a matmul whose placeholder and whose .packed sibling both carry no bytes, and the decision moved to plan() — which is the one place that knows whether a Token .fconst exists. Absent one, the refusal stands (with #1687's wording, by name); with one, the node is dropped and the weights come from the .fconst, which is your reading of the pack and the correct one.

    One half of this issue is still open, and it is not #1688's: the self-check corr 0.999999 you quote as the pre-#1686 behaviour is currently 0.885220 on main, for a reason unrelated to the throw — #1661 repointed embed_bytes at the container's bf16 copy and left four readers decoding F16. The weights are right (I compared the containers tensor by tensor: all 291 identical); only the number about them is wrong. #1690 restores it and is READY: on its head the same import reports corr 0.999999 rel_err 0.000700 and the pack leg returns rc 0.

    Closing as fixed for the refusal; #1690 carries the diagnostic half and is worth a merge if anything reads that line.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions