Skip to content

npu: Tensor::avail is measured to the end of the file, so three bounds miss a read into the next tensor #1881

Closed

Description

@bong-water-water-bong

Tensor::avail is measured to the end of the file, so three bounds that use it miss a read into the next tensor

Found while fixing #1843 (PR #1878), and asked to be filed separately in that PR's review.

npu/model.cpp sets each tensor's avail to size_ - base - off[0]: the bytes from the tensor's start to the end of the file. The tensor's own length is bytes = off[1] - off[0]. Three bounds on main check a read against avail:

npu/lane.cpp:101   if (norm.avail < size_t(d.hidden) * 2)                       -> "model.norm.weight runs past the end of the file"
npu/lane.cpp:103   if (model.tensor("model.embed_tokens.weight").avail < size_t(d.vocab) * size_t(d.hidden) * 2)
npu/pack.cpp:70    if (bytes > t.avail) throw ... " runs past the end of the file"

Each one catches a read that runs past the file. None catches a read that runs past the tensor into whatever is laid out after it, which is every tensor except the last. A config that over-states a dimension (vocab, hidden, a projection's shape) then reads the next tensor's bytes as its own, quietly.

Scope:

Fix: compare against bytes (the tensor) and keep avail only as the outer file bound. Or drop avail from these three sites. The npu_model_bounds_test fixture can lay out two tensors back to back and over-state the first, so an arm shows each site refusing where today it reads into the neighbour. No device is needed.

Activity

  1. bong-water-water-bong commented on Nov 10, 2026

    @bong-water-water-bong
    CollaboratorAuthor

    Claimed by pi-bonsai, 2026-11-10 ~06:1xZ. Taking the general fix rather than the embedding half alone, since #1880 is this same defect seen from one of the three sites.

    Approach exactly as scoped here — the bound becomes the tensor, with the file bound kept as the outer limit:

    // npu/model.h
    uint64_t avail = 0;  // bytes from data to the end of the file: the outer bound a reader may not cross
    uint64_t usable() const { return bytes < avail ? bytes : avail; }   // engine#1881

    All three sites switch to usable(): npu/lane.cpp:101 (final norm), npu/lane.cpp:103 (embedding table), npu/pack.cpp:70 (need(), every projection pack). Both halves of the min are load-bearing: avail alone misses a read into the next tensor, bytes alone misses a malformed container that declares more than the file holds.

    Arms go into tests/npu_model_bounds_test.cpp (already includes model.h and is already in ci.yml's --target list, so no new target and no workflow scope needed). The arm that bites is the middle one — avail 4096 > bytes 64 must give 64 — because a reverted avail check passes it.

    PR to follow when the test is green.

  2. bong-water-water-bong commented on Nov 10, 2026

    @bong-water-water-bong
    CollaboratorAuthor

    Recording the residual after #1883, from reviewing its follow-up #1890 (Ready).

    #1883 bound reads by the tensor (bytes) instead of the file — but it kept a bf16 expected size, so it silently refused every quantised embedding table: bonsai2-27b-NPU's I8 tiled table is 794,624,000 B against a vocab × hidden × 2 of 2,542,796,800 B (a 5120-byte tile covers 32 × 256 = 8,192 weights, i.e. 0.625 B/weight, which is where 794,624,000 comes from exactly). #1890 derives the expected size from the dtype and fixes that.

    What remains open, scoped and deliberate: for an I8 chunk grid the guard falls back to the file-end bound (avail). A chunk grid's shape product is already a byte count (155,200 "rows" are 7,760 row tiles × 20 column tiles), so comparing it against V × H was the bug; size-checking it properly needs the tile arithmetic only the forward has. The consequence is that this issue's original hole — an overstated vocab passing because another tensor follows — is still open for quantised tables, and only closed for float ones.

    So the honest status of this issue after #1883 + #1890 is: half-fixed by construction. Worth stating in the issue rather than only in #1890's code comment, so the next person to hit it does not re-file the original bug, and so anyone adding a precise I8 check knows exactly which case they are completing.

    One request on #1890 while it is open: its arms are unit tests of expected_bytes() (bf16/f16/f32/i32/i8/u8/unknown/zero) and do not drive the guard with a quantised table. Since the mistake being fixed was which argument the caller passes, a container-level arm (an I8 table of the bonsai2 shape is not refused) is the one that would fail on main today — the unit arms cannot distinguish a caller that passes V × H from one that passes the chunk-grid product.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions