Skip to content

npu-pack: the external-data projection lookup concatenates without a dot, and the container still reports complete; the implicit-zero-point block is duplicated with #1794 #1841

Closed

Description

@bong-water-water-bong

Found auditing the merges that went in over an unanswered CHANGES (~/briefs/MERGED-OVER-CHANGES-2026-11-10.md, row #1790). Two separate points, both still open on main.

1. The container reports itself complete while no layer projection was imported

The CHANGES named this at merged head 87a0dcf: the walker change is right (accept 3 inputs, zeros only when present) and the head correlation is real (0.993981 against
the pack's own embedding, both controls failing) — but every projection lookup missed, and the container said it was complete.

Cause, quoted from the review: decode_rows falls back to

init_of(proj + "MatMul.weight_Q4")        // proj does not end in '.'
init_of(proj + "MatMul.weight_scales")

so a proj without the trailing dot builds …o_projMatMul.weight_Q4 instead of …o_proj.MatMul.weight_Q4. Nothing throws; the lookup misses, the tensor is skipped, and
"complete" is computed from what was walked rather than from what the pack declares.

Severity: this is the same class as the tie_word_embeddings default and the block-gate refusal — a pack that imports and serves without the weights it needs. A
gaps 0 report over a container with no layer projections is worse than a refusal, because it reads as a finished import.

2. The same change is inside #1794, and both merged

The second CHANGES measured it: both diffs add the same block at the same position in npu/pack_import.cpp (// implicit zero point, which implicit_zp_zero() below applies. and the ── The implicit zero point ── header), so merging both applies the work twice. Whether that is currently harmful depends on whether the block is
idempotent — a duplicated comment is cosmetic, a duplicated table fill is not — and that has not been checked.

What a fix looks like

  1. the dot: build the key with an explicit separator rather than string concatenation, and make a lookup miss count as a gap so gaps 0 cannot be reported over a
    pack with no projections;
  2. the duplicate: diff the two copies as they are on main and delete one, or prove the block idempotent and say so where it is defined.

Not a revert candidate: both halves are forward fixes, and the pack in question (~/scratch-packs/qwen3vl4b.q4nx) is the reproduction for the first.

Activity

  1. bong-water-water-bong commented on Nov 10, 2026

    @bong-water-water-bong
    CollaboratorAuthor

    Claimed this to fix it; two of the three premises are already stale on main, so here is the verified state rather than a PR that would
    re-fix the fixed half.

    1. The dot: fixed, and the fix cites the same failure

    decode_rows now normalises before appending a suffix (npu/pack_import.cpp:3677):

    std::string pre = proj;
    if (!pre.empty() && pre.back() != '.') pre += '.';
    const OnnxTensor* qw = init_of(pre + "MatMulNBits.qweight");

    with the comment naming exactly the mechanism the issue describes — "otherwise the three-input spelling is looked up as ...q_projMatMul.weight_Q4
    and never found, which is how a pack could plan all 252 projections, write only its norms, and fail on the device with MLP dequant : data_size 0"
    .
    It arrived with a merged PR rather than as a separate fix. So the gaps 0-over-no-projections symptom should be re-checked on main before anyone
    writes another patch for it; if it still reproduces with the dot present, that is a different bug and the reproduction (~/scratch-packs/qwen3vl4b.q4nx)
    is what would show it.

    2. The implicit zero point: not applied twice — duplicated instead, with two different bits

    Both call sites are on main, and they are in different code paths, so no tensor is charged twice:

    :3718   if (!zt)     implicit_zp_zero(q, 4);          // projection / external-data path, bits HARDCODED
    :3840   if (!have_zp) implicit_zp_zero(q, mm.bits);   // graph-walk path, bits from the node's attribute
    

    Neither is wrong today — the projection path only matches the weight_Q4 spelling, so 4 is implied by the name — but the rule now exists twice with
    two different derivations of the same parameter, and nothing asserts that the Q4 spelling and the literal 4 agree. That is a drift risk, not a
    double application: a later reader adding a weight_Q8 spelling would change the lookup and leave the zero point at 8 (2^3), silently.

    The cheap consolidation, if anyone wants it: one implicit_zp_zero_for(Q4&, bits, const char* why) taking bits from the spelling it matched, with
    a comment saying a spelling that implies other than 4 bits must pass its own value. I am not opening that as a PR tonight — it is a
    tidiness change with no behaviour difference on any pack we have, and the fleet has more pressing rows.

    3. What I did not verify

    Whether the gaps report counts a lookup miss as a gap, which was point 1's second half ("make a lookup miss count as a gap so gaps 0 cannot be
    reported over a pack with no projections"). With the dot fixed, a miss is no longer silent for that spelling — but the general question (does
    "complete" derive from what was walked or from what the pack declares?) is worth one look by whoever owns the report, and I have not done it.

  2. bong-water-water-bong commented on Nov 10, 2026

    @bong-water-water-bong
    CollaboratorAuthor

    Point 3 verified on main, point 2 now in a PR (#1893).

    Point 3 is closed by code, not by intent. 486c8cbc made every way out of decode_rows() a named gap:

    // Every way out of here that writes nothing is a named gap (engine#1841), never a quiet skip.
    auto gap = [&](const std::string& why) { proj_gaps.push_back(out_name + ": \`" + pre + "\` " + why); return false; };
    ...
    if (!qw || !sc) return gap("has no quantised projection the decoder can pair (tried MatMulNBits.qweight + .scales and "
                               "MatMul.weight_Q4 + _scales)");

    and the caller turns proj_gaps into r.refused and refuses the container unless opt.allow_incomplete. So a pack whose projections were never written can no longer report gaps 0 — the miss is a named refusal, which is exactly what that half of the issue asked for.

    Point 2 is now #1893 rather than left as a note: zp_bits_for_spelling() derives the width from the suffix that matched (weight_Q4 → 4, weight_Q8 → 8) and falls back to the node's value for a spelling with no _Q<d>, so the two call sites agree by construction. Six arms pin it in the import test — including the one that fails under a hardcoded literal (weight_Q8 must give 8, not 4) — and the Qwen3-VL-4B import is unchanged after the patch (399 tensors, 491,000 tiles, corr 0.993981), because that pack spells weight_Q4.

    Context on the premise that motivated the issue: the gaps 0 symptom itself was real and is fixed — re-verified today on a main build (3f2f6afe), where the same pack imports 399 tensors / 491,000 tiles and serves on the device with three identical The capital of France is **Paris**. answers. So the audit trail here ends with the symptom gone, the miss path named, and the duplicated rule consolidated.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions