Skip to content

npu: the tiled embedding path is chosen on a divisibility test alone, so an unsupported chunk kind answers zeros instead of refusing #1842

Closed

Description

@bong-water-water-bong

The tiled embedding read can produce silent zeros instead of a load-time refusal

Found auditing #1807 (merged 54a259f) against the CHANGES at its merged head (this row of MERGED-OVER-CHANGES-2026-11-10.md). The review's
blocking point was never addressed, and the code on main is unchanged from the reviewed head.

Where: npu/forward/src/q4nx_forward.cpp:745-756

const int ntr = NV / 32, ntc = kpad_ / 256;
const size_t tiles = (size_t)ntr * (size_t)ntc;
if (tiles > 0 && esz % tiles == 0) {
    embed_tile_       = (const uint8_t*)e;
    embed_tile_bytes_ = esz;
    embed_tile_rows_  = NV;
    embed_tile_cols_  = kpad_;
    embed_tile_cb_    = int(esz / tiles);   // <- never validated as a chunk kind
    ...
}

The tiled path is taken on esz % tiles == 0 alone. q4nx_dequant_tile_row() then refuses at runtime for conditions init never checked:

  • chunk_bytes is not one of Q4NX_TILE_BYTES / Q4NX_CHUNK_Q4K / Q4NX_CHUNK_Q8;
  • NV % 32 != 0 (so ntr = NV / 32 truncates and the tile grid does not cover the table);
  • kpad_ % 256 != 0 (same, across columns).

Those refusals return -1, and embed_at() maps -1 to 0.0f — so a pack that satisfies the divisibility test but has an unsupported chunk
kind loads successfully and answers with zeros, with no message. That is the failure mode §2 rules out ("a gate reads the artifact under the real
runtime condition… never a label"), and it is worse than a refusal because the output looks like a plausible model answer.

What it needs (any one of these, whichever the forward's owner prefers):

  1. validate at load: before setting embed_tile_, require NV % Q4NX_TILE_ROWS == 0, kpad_ % Q4NX_TILE_COLS == 0, and
    esz / tiles to be one of the three supported chunk kinds — otherwise leave embed_tile_ null so the whole-table path runs (the existing fallback),
    or set err_ and refuse if the buffer path also cannot handle the dtype;
  2. or make embed_at() distinguish "no element-wise read" from "the read failed", so a -1 cannot be served as 0.0f;
  3. or if a silent zero is somehow correct for these pack shapes, say so at the call site — but nothing in the PR or the code suggests it is.

Reproduction without a device: the three conditions are pure arithmetic on NV, kpad_ and esz. tests/npu_dequant_bf16_test.c already carries
the chunked row arms that pin the three valid kinds and five refusals; the missing arm is the init-side one: a table whose esz % tiles == 0 but whose
chunk kind is unsupported must not select the tiled path.

Not asking for a revert — the tiled read is a real improvement (0.74 GiB packed instead of 4.74 GiB of f32) and the happy path is byte-exact. This is the
missing guard on the path selection.

Activity

  1. bong-water-water-bong commented on Nov 10, 2026

    @bong-water-water-bong
    CollaboratorAuthor

    Confirmed exactly as filed, and fixed: #1875 (1bit/1842-tile-probe, 8577203ac) — a cherry-pick -x of the commit that the merge raced past.

    You are right that my review's blocking point "was never addressed, and the code on main is unchanged from the reviewed head". The fix is 0c6e163e9 on 1bit/1784-chunked-row; the merge landed 38170f8, the head before it, so the commit is unreachable from main while the branch is closed. Verified rather than assumed:

    $ git show origin/main:npu/forward/src/q4nx_forward.cpp | grep -c "cb, 0, probe.data()"     → 0
    $ git show 0c6e163e9:npu/forward/src/q4nx_forward.cpp   | grep -c "cb, 0, probe.data()"     → 1
    

    So: not a disagreement about the fix, a merge race. The re-land carries the same probe (row 0 and row NV-1; a failure leaves embed_tile_ null and the unchanged buffer path refuses with its own message), and the same measurement I posted on the PR: 9932 → 2619 MiB RssAnon on the 27B with NPU_FFN_CACHE pinned.

    What I am taking from this: a verdict belongs to a head. When a blocking point is fixed after the approving review, the fix has to be what merges, or it needs its own PR immediately — I fixed it, re-measured it, posted it, and then trusted a merge that had already happened. Your audit row is the only reason it was caught, and the same check found #1839 the same night (a real 18× quantiser defect). Two defects, both through a green build and an approving review; the rule is worth keeping exactly as written.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions