You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
npu: Tensor::avail is measured to the end of the file, so three bounds miss a read into the next tensor #1881
Tensor::avail is measured to the end of the file, so three bounds that use it miss a read into the next tensor
Found while fixing #1843 (PR #1878), and asked to be filed separately in that PR's review.
npu/model.cpp sets each tensor's avail to size_ - base - off[0]: the bytes from the tensor's start to the end of the file. The tensor's own length is bytes = off[1] - off[0]. Three bounds on main check a read against avail:
npu/lane.cpp:101 if (norm.avail < size_t(d.hidden) * 2) -> "model.norm.weight runs past the end of the file"
npu/lane.cpp:103 if (model.tensor("model.embed_tokens.weight").avail < size_t(d.vocab) * size_t(d.hidden) * 2)
npu/pack.cpp:70 if (bytes > t.avail) throw ... " runs past the end of the file"
Each one catches a read that runs past the file. None catches a read that runs past the tensor into whatever is laid out after it, which is every tensor except the last. A config that over-states a dimension (vocab, hidden, a projection's shape) then reads the next tensor's bytes as its own, quietly.
lane.cpp:101 (the final norm against hidden) and pack.cpp:70 (every projection pack) have no other check.
Fix: compare against bytes (the tensor) and keep avail only as the outer file bound. Or drop avail from these three sites. The npu_model_bounds_test fixture can lay out two tensors back to back and over-state the first, so an arm shows each site refusing where today it reads into the neighbour. No device is needed.
Claimed by pi-bonsai, 2026-11-10 ~06:1xZ. Taking the general fix rather than the embedding half alone, since #1880 is this same defect seen from one of the three sites.
Approach exactly as scoped here — the bound becomes the tensor, with the file bound kept as the outer limit:
// npu/model.huint64_t avail = 0; // bytes from data to the end of the file: the outer bound a reader may not crossuint64_tusable() const { return bytes < avail ? bytes : avail; } // engine#1881
All three sites switch to usable(): npu/lane.cpp:101 (final norm), npu/lane.cpp:103 (embedding table), npu/pack.cpp:70 (need(), every projection pack). Both halves of the min are load-bearing: avail alone misses a read into the next tensor, bytes alone misses a malformed container that declares more than the file holds.
Arms go into tests/npu_model_bounds_test.cpp (already includes model.h and is already in ci.yml's --target list, so no new target and no workflow scope needed). The arm that bites is the middle one — avail 4096 > bytes 64 must give 64 — because a reverted avail check passes it.
Recording the residual after #1883, from reviewing its follow-up #1890 (Ready).
#1883 bound reads by the tensor (bytes) instead of the file — but it kept a bf16 expected size, so it silently refused every quantised embedding table: bonsai2-27b-NPU's I8 tiled table is 794,624,000 B against a vocab × hidden × 2 of 2,542,796,800 B (a 5120-byte tile covers 32 × 256 = 8,192 weights, i.e. 0.625 B/weight, which is where 794,624,000 comes from exactly). #1890 derives the expected size from the dtype and fixes that.
What remains open, scoped and deliberate: for an I8 chunk grid the guard falls back to the file-end bound (avail). A chunk grid's shape product is already a byte count (155,200 "rows" are 7,760 row tiles × 20 column tiles), so comparing it against V × H was the bug; size-checking it properly needs the tile arithmetic only the forward has. The consequence is that this issue's original hole — an overstated vocab passing because another tensor follows — is still open for quantised tables, and only closed for float ones.
So the honest status of this issue after #1883 + #1890 is: half-fixed by construction. Worth stating in the issue rather than only in #1890's code comment, so the next person to hit it does not re-file the original bug, and so anyone adding a precise I8 check knows exactly which case they are completing.
One request on #1890 while it is open: its arms are unit tests of expected_bytes() (bf16/f16/f32/i32/i8/u8/unknown/zero) and do not drive the guard with a quantised table. Since the mistake being fixed was which argument the caller passes, a container-level arm (an I8 table of the bonsai2 shape is not refused) is the one that would fail on main today — the unit arms cannot distinguish a caller that passes V × H from one that passes the chunk-grid product.
Activity
bong-water-water-bong commentedon Nov 10, 2026
Claimed by pi-bonsai, 2026-11-10 ~06:1xZ. Taking the general fix rather than the embedding half alone, since #1880 is this same defect seen from one of the three sites.
Approach exactly as scoped here — the bound becomes the tensor, with the file bound kept as the outer limit:
All three sites switch to
usable():npu/lane.cpp:101(final norm),npu/lane.cpp:103(embedding table),npu/pack.cpp:70(need(), every projection pack). Both halves of theminare load-bearing:availalone misses a read into the next tensor,bytesalone misses a malformed container that declares more than the file holds.Arms go into
tests/npu_model_bounds_test.cpp(already includesmodel.hand is already inci.yml's--targetlist, so no new target and no workflow scope needed). The arm that bites is the middle one —avail4096 >bytes64 must give 64 — because a revertedavailcheck passes it.PR to follow when the test is green.
test: pin Tensor::expected_bytes' widths, the table the embedding gua…
bong-water-water-bong commentedon Nov 10, 2026
Recording the residual after #1883, from reviewing its follow-up #1890 (Ready).
#1883 bound reads by the tensor (
bytes) instead of the file — but it kept a bf16 expected size, so it silently refused every quantised embedding table:bonsai2-27b-NPU's I8 tiled table is794,624,000 Bagainst avocab × hidden × 2of2,542,796,800 B(a 5120-byte tile covers32 × 256 = 8,192weights, i.e. 0.625 B/weight, which is where794,624,000comes from exactly). #1890 derives the expected size from the dtype and fixes that.What remains open, scoped and deliberate: for an I8 chunk grid the guard falls back to the file-end bound (
avail). A chunk grid's shape product is already a byte count (155,200 "rows"are 7,760 row tiles × 20 column tiles), so comparing it againstV × Hwas the bug; size-checking it properly needs the tile arithmetic only the forward has. The consequence is that this issue's original hole — an overstatedvocabpassing because another tensor follows — is still open for quantised tables, and only closed for float ones.So the honest status of this issue after #1883 + #1890 is: half-fixed by construction. Worth stating in the issue rather than only in #1890's code comment, so the next person to hit it does not re-file the original bug, and so anyone adding a precise I8 check knows exactly which case they are completing.
One request on #1890 while it is open: its arms are unit tests of
expected_bytes()(bf16/f16/f32/i32/i8/u8/unknown/zero) and do not drive the guard with a quantised table. Since the mistake being fixed was which argument the caller passes, a container-level arm (an I8 table of the bonsai2 shape is not refused) is the one that would fail onmaintoday — the unit arms cannot distinguish a caller that passesV × Hfrom one that passes the chunk-grid product.