You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
npu-pack: the external-data writer has no success-path test — every import_pack() arm on that genre expects a refusal #1849
Found auditing the merges that went in over an unanswered CHANGES (~/briefs/MERGED-OVER-CHANGES-2026-11-10.md, row #1712).
What is missing
#1712 (merged f0ccb26b) re-landed the external-data tile decoder, and its 2060191 added the writer (+158 lines in npu/pack_import.cpp). The CHANGES at the merged head b3da416 accepted the decoder, which has per-row scale and zero arms that fail under mutation, and blocked on one thing: nothing imports an external-data pack to completion and checks what lands in the container.
That is still true on main (ef865f9f):
The only import_pack() call on this genre (tests/npu_pack_import_test.cpp, the eager-mm fixture around :540) expects a refusal ("import refuses a pack whose geometry no split can satisfy") and asserts no model.q4nx is written.
No test calls or reaches import_external on a success path (git grep import_external origin/main -- tests gives 0 hits).
So three merged changes to the writer have run only against real packs on the box, and a regression in what it writes (tile order, scale placement, the header keys) would pass CI.
The fix
One crafted external-data pack whose geometry the container split can satisfy (the eager-mm fixture with a fitting qkv width for its head count), imported to completion. Assert:
model.q4nx exists, and the report has no refusals;
one tile read back from the container decodes to the codes, bf16 scales and int4 zeros the fixture wrote (the decoder arm's distinct-per-row values, so a row mix-up fails);
Claimed by pi-bonsai, 2026-11-10 ~07:1xZ. Taking the fix as written: one crafted external-data pack whose geometry the container split can satisfy, three arms (container exists + no refusals; one tile read back decodes to the codes / bf16 scales / int4 zeros the fixture wrote; the header keys the later fixes added), and the mutation the issue asks for — swap two rows' scales in the writer and arm 2 must fail — as the proof that arm 2 bites.
I reviewed #1712's family (and #1751's and #1793's writer changes), so the risk this closes is one I have been sharing: three merged changes to import_external are exercised only by real packs on the box, and nothing in CI would notice a change in what it writes.
Scoped the fixture; two things a naive version of this test would get wrong, recorded so they are not discovered twice.
1. The existing external-packed fixture's data is unusable for arm 2 by itself
tests/npu_pack_import_test.cpp writes the packed blob as all 0x22 (write_data(stride * N)), and the fitting case at :445 already gets external_row_stride {"matched":1,"of":1}. But a 68-byte external block is 64 B of codes + 4 B (pack_import.cpp:329: "N x (K/128+1) x 68"), so all-0x22 means every nibble is 2, the zero point is 2 and the reconstructed weight is (2−2)·s = 0 for every element of the tensor.
So an arm that asserts "one tile read back decodes to the fixture's values" would assert all zeros — and an all-zero decode is exactly what a broken writer produces too. That is the failure #1871 and #1875 spent tonight closing (a table that serves 0.0f for every read, silently). The fixture needs crafted bytes, not x22 repeated.
2. What the crafted blob should be
Per 128-K block: nibbles 0x9A alternating (codes 9, 10), zero point 8, scale bf16 1.0 (0x3F 0x80) → decoded weights 1.0, 2.0, 1.0, 2.0 …: non-zero, distinct, and not equal to the zero point, so arm 2 fails if the writer drops the codes, the scale or the zero. One row of the fixture then has a value a human can predict from the bytes.
3. What I have confirmed for the write/read path
pi::import_pack(dir, out, pi::Options{allow_incomplete}) is the entry (tests/npu_pack_import_test.cpp:786), and the genre's success path is not reached by any arm today (git grep import_external -- tests = 0).
q4nx_dequant_tile_row(data, nbytes, rows, cols, chunk_bytes, r, out) (npu/forward/include/q4nx_dequant.h:91) is the read-back for arm 2 — it returns "the values that function would place at that row, bit for bit", so the comparison is exact rather than approximate.
Claimed, scoped, not yet written — the three arms plus the "swap two rows' scales in the writer must fail arm 2" mutation are the plan, and I would rather land them together than ship arm 1 alone (a container that exists and does not throw is precisely the kind of green that says nothing). If another lane wants it sooner, the fixture change is the whole task: crafted bytes, one import_pack call, one q4nx_dequant_tile_row.
Progress: the hard half is done and measured, the read-back mapping is the open piece. Committed locally (3d8a0e99 on npu-pack-external-success in ~/wt/1862-npurun), not pushed — arm 2 is not assertable yet and I will not redden CI with a test that fails for a reason I have not explained.
What works now
The fixture geometry was the blocker, and it is the issue's own phrase. The existing external-packed fixture is 768 wide, and the container split refuses it:
model.layers.0.attn.qkv_proj.MatMulNBits.qweight.packed: container range [0, 5120) does not fit N 768
So no arm had ever reached the writer. With 40 heads / 40 kv heads / head_dim 128 the fused qkv is 3 × 5120 = 15360 wide, every role's slice is 5120, and pi::import_packsucceeds:
writer success: model.layers.0.self_attn.q_proj.weight 19,660,800 B (header 576 B)
The crafted blob (per the earlier comment): nibbles 0xCC → int4 −4, zero point 0x11 → four 1s, scale bf16 0x4000 → 2.0, so (c−z)·s = (−4−1)·2 = −10.0 — one value that fails if the writer drops the codes (−2.0), the zero point (−8.0) or the scale (−5.0). Not the all-0x22 the other fixtures write, which decodes to (2−2)·s = 0 and cannot distinguish a broken writer from a correct one.
CMake: npu/forward/src/q4nx_dequant.c + npu/forward/include added to the existingnpu_pack_import_test target — sources on a target already in ci.yml's --target list, so no new test and no workflow scope.
The open piece, with the measurement that localises it
The written tensor keeps shape 3200 × 6144 (padded), q4nx_dequant_tile_row returns 0 (it decodes), and sampling rows:
row 0: first 0.0000 mid 0.0000
row 32: first 0.0000 mid 0.0000
row 160: first 0.0000 mid 0.0000
row 3199: first 26.0000 mid 26.0000
So the writer did consume the blob (the last row is non-zero and the values are not padding), but the container's row index is not the tensor's row index — either tile-blocked order, or the padded rows sit at the front of the tile grid. 26.0 is also not yet explained by my byte construction, which is a second reason to read packB_blocked / write_container's row walk rather than infer it: with 6144 columns (5120 padded to a 256 multiple) the block offsets differ from the 68-byte stride I crafted.
Next step is mechanical once that walk is read: decode the row the writer would place the first real row at, assert −10.0, then run the mutation the issue asks for (swap two rows' scales in the writer; arm 2 must fail). I would rather hand over a measured localisation than a test that is green for the wrong reason.
Two more measured facts, both of which change what the read-back has to do (local commit +1 on npu-pack-external-success).
1. The stored shape is [tile count, bytes per tile], not [rows, cols].pack_import.cpp writes w.add(prefix, "I8", {nt, kImportTileBytes}, …) with kImportTileBytes = q4nx::Chunk::Q4_1F = 6144. So my q_proj's 19,660,800 B is 3200 tiles × 6144 B, and "shape 3200 × 6144" was that pair — which explains the 0.0 / 26.0 mixture I reported earlier: I was indexing rows against a 3200-row grid that does not exist.
2. The read-back has to take the device loader route, and it did not fire.q4nx_dequant_tile_row() decodes only the three 5120/Q4K/Q8 chunk kinds and refuses 6144 outright (−1). q4nx_device.cpp:263-266 handles this by repacking first: as_q4_1(x) then decode 5120-B tiles. I called q4nx::as_q4_1 on {I8, shape {5120,5120}, bytes 19660800} and got 19,660,800 B back unchanged — a pass-through branch rather than the 3200 × 5120 = 16,384,000 B repack.
So the next step is narrow and mechanical: read what as_q4_1 (npu/q4nx.cpp:210) dispatches on — the caller has to satisfy its precondition for the conversion to happen — then assert −10.0 on the first real row and run the mutation. The fixture, the import, the geometry and the crafted blob are all in place and green; only the tile-read-back chain is unresolved. Still not pushed, for the same reason as before.
The read-back chain works end to end and the suite is green (local commit 01c90006, still not pushed). Three findings, the middle one a correction of my own.
1. chunk_of() reads shape.back() — which is why the writer stores {ntiles, chunk_bytes}. Giving the Tensor {rows, cols} made chunk_of() see 5120 (Q4_1) and as_q4_1() copied the bytes straight through; giving it {3200, 6144} made it repack 19,660,800 → 16,384,000 B (3200 tiles) and the rows decode. So the stored shape is not an accident, it is the chunk tag.
2. The value is 22.0, not the −10.0 I asserted — and my first derivation was wrong, not the writer. MatMulNBits codes are unsigned, with the zero point shifting them: w = (nibble − zp) · scale = (12 − 1) · 2.0 = 22.0. pack_import.cpp says so itself (w = (code_nibble - zp_nibble) * scale). I had read nibble 12 as int4 −4; every row decoding to a clean 22.0 is what exposed it.
3. The mutation is caught by the suite but not by my arm. Doubling one scale slot in container_from_external_q4 fails the suite (exit 1):
FAIL external->container: d is the pack scale
FAIL external->container: code*d + m reproduces (code - zp)*scale to within one bf16 ulp of m
but my "every row decodes to 22.0" arm stays green — so it is less sensitive than the sibling arms on this writer, and I have not explained why the doubled slot does not reach the q_proj values it decodes. I made the arm scan every row (a row-0-only check is blind to a wrong slot anywhere else) and it still does not fire.
That is why this is still local: the issue asks that arm 2 fail on the mutation, and it does not. What the work does establish is that the fixture, the satisfying geometry, the import, the container and a value-level read-back all exist and are green — so the remaining question is narrowly about where the doubled slot is lost, not about whether a success-path test is possible.
Done — PR #1902. The mutation now bites the arm, and the mechanism explains why my first version missed it.
The mutation (double one scale slot in container_from_external_q4) fails it as:
FAIL external writer: every one of k_proj 5120 rows decodes to 22.0 ((12 - 1) * 2.0);
32 differ, first bad row 2560
The reason my q-only arm stayed green: container_from_external_q4() runs on the FUSED qkv, so scale[size()/2] of a 15360-row pack lands in k_proj row 2560 — exactly half of 5120 — and q_proj is untouched. The arm now checks q, k and v, which is where the writer actually works.
Also settled since the last comment: the value is 22.0 ((nibble − zp)·scale, MatMulNBits codes are unsigned — pack_import.cpp says so in its own comment), and the read-back is the device loader's route (chunk_of() reads shape.back(), so the Tensor is shaped {ntiles, chunk_bytes}; as_q4_1() repacks 19,660,800 → 16,384,000 B and the rows decode). No second decoder was written for the test.
Whole suite green with the writer intact; npu/forward/src/q4nx_dequant.c is added to the existingnpu_pack_import_test target, so no new test target and no workflow scope.
Two PRs add the missing import_external() success path — map, and a decision
Both PRs exist because of the same gap, and both describe it identically in substance: every existing import_pack() arm on the 1.9 external-data genre expected a refusal, so the writer, its BF16 copies and the #1751/#1793 header keys had only ever run against real packs.
crafts a pack, imports it, then inspects the container: blob size follows the layout arithmetic (N*(K/128+1)*68), why/refused empty andmodel.q4nx exists, num_attention_heads/num_key_value_heads match, q4nx_weight_layout is b8 where expected and absent where a default must not emit it, tokenizer.json copied, per-tensor (hi−lo) == RT·KT·tb, plus the newer int4 sign-extend reference (all 16 nibbles, `
crafts a pack and asserts the import succeeds, model.q4nx exists, the container carries the expected role files, and a bad == 0 count
Recommendation: land #1863, close #1902 as covered.#1863 asserts the writer's contract (the sizes and keys a reader must find), while #1902 asserts that the path runs and produces a container — a smoke test. Both are good, but only one can land in this test, and the one with ten assertions subsumes the other's four.
What #1902 has that #1863 should borrow before #1902 closes: the role-file assertion (the container carries <role>) and the aggregate bad == 0 count. #1863 checks the token table and the config, but a multi-component genre's role files are exactly what a future multi-component pack will depend on — one more assertion in #1863 and nothing from #1902 is lost.
And one thing to keep visible when they close: this is the third cluster tonight where the same gap drew two or more PRs (#1889 → #1895/#1896/#1897; #1864 → #1891/#1898/#1866; this). The pattern is that a CHANGES-driven requirement is discovered by several lanes at once from the same audit row — worth the lanes checking gh pr list --search for the issue number before starting, which is the rule the brief already states for main-is-red fixes.
Activity
bong-water-water-bong commentedon Nov 10, 2026
Claimed by pi-bonsai, 2026-11-10 ~07:1xZ. Taking the fix as written: one crafted external-data pack whose geometry the container split can satisfy, three arms (container exists + no refusals; one tile read back decodes to the codes / bf16 scales / int4 zeros the fixture wrote; the header keys the later fixes added), and the mutation the issue asks for — swap two rows' scales in the writer and arm 2 must fail — as the proof that arm 2 bites.
I reviewed #1712's family (and #1751's and #1793's writer changes), so the risk this closes is one I have been sharing: three merged changes to
import_externalare exercised only by real packs on the box, and nothing in CI would notice a change in what it writes.bong-water-water-bong commentedon Nov 10, 2026
Scoped the fixture; two things a naive version of this test would get wrong, recorded so they are not discovered twice.
1. The existing
external-packedfixture's data is unusable for arm 2 by itselftests/npu_pack_import_test.cppwrites the packed blob as all0x22(write_data(stride * N)), and the fitting case at :445 already getsexternal_row_stride {"matched":1,"of":1}. But a 68-byte external block is64 B of codes + 4 B(pack_import.cpp:329: "N x (K/128+1) x 68"), so all-0x22means every nibble is 2, the zero point is 2 and the reconstructed weight is(2−2)·s = 0for every element of the tensor.So an arm that asserts "one tile read back decodes to the fixture's values" would assert all zeros — and an all-zero decode is exactly what a broken writer produces too. That is the failure #1871 and #1875 spent tonight closing (a table that serves
0.0ffor every read, silently). The fixture needs crafted bytes, notx22repeated.2. What the crafted blob should be
Per 128-K block: nibbles
0x9Aalternating (codes 9, 10), zero point8, scale bf161.0(0x3F 0x80) → decoded weights 1.0, 2.0, 1.0, 2.0 …: non-zero, distinct, and not equal to the zero point, so arm 2 fails if the writer drops the codes, the scale or the zero. One row of the fixture then has a value a human can predict from the bytes.3. What I have confirmed for the write/read path
pi::import_pack(dir, out, pi::Options{allow_incomplete})is the entry (tests/npu_pack_import_test.cpp:786), and the genre's success path is not reached by any arm today (git grep import_external -- tests= 0).q4nx_dequant_tile_row(data, nbytes, rows, cols, chunk_bytes, r, out)(npu/forward/include/q4nx_dequant.h:91) is the read-back for arm 2 — it returns "the values that function would place at that row, bit for bit", so the comparison is exact rather than approximate.q4nx_weight_layout(arm 3): present only for a non-default layout, which is the negative direction npu-pack: pin q4nx_weight_layout being written, in both directions (engine#1840) #1877 pinned at the importer — the same key, so the two arms should share a fixture.Status
Claimed, scoped, not yet written — the three arms plus the "swap two rows' scales in the writer must fail arm 2" mutation are the plan, and I would rather land them together than ship arm 1 alone (a container that exists and does not throw is precisely the kind of green that says nothing). If another lane wants it sooner, the fixture change is the whole task: crafted bytes, one
import_packcall, oneq4nx_dequant_tile_row.bong-water-water-bong commentedon Nov 10, 2026
Progress: the hard half is done and measured, the read-back mapping is the open piece. Committed locally (
3d8a0e99onnpu-pack-external-successin~/wt/1862-npurun), not pushed — arm 2 is not assertable yet and I will not redden CI with a test that fails for a reason I have not explained.What works now
The fixture geometry was the blocker, and it is the issue's own phrase. The existing
external-packedfixture is 768 wide, and the container split refuses it:So no arm had ever reached the writer. With 40 heads / 40 kv heads / head_dim 128 the fused qkv is 3 × 5120 = 15360 wide, every role's slice is 5120, and
pi::import_packsucceeds:The crafted blob (per the earlier comment): nibbles
0xCC→ int4 −4, zero point0x11→ four 1s, scale bf160x4000→ 2.0, so(c−z)·s = (−4−1)·2 = −10.0— one value that fails if the writer drops the codes (−2.0), the zero point (−8.0) or the scale (−5.0). Not the all-0x22the other fixtures write, which decodes to(2−2)·s = 0and cannot distinguish a broken writer from a correct one.CMake:
npu/forward/src/q4nx_dequant.c+npu/forward/includeadded to the existingnpu_pack_import_testtarget — sources on a target already inci.yml's--targetlist, so no new test and no workflow scope.The open piece, with the measurement that localises it
The written tensor keeps shape 3200 × 6144 (padded),
q4nx_dequant_tile_rowreturns 0 (it decodes), and sampling rows:So the writer did consume the blob (the last row is non-zero and the values are not padding), but the container's row index is not the tensor's row index — either tile-blocked order, or the padded rows sit at the front of the tile grid.
26.0is also not yet explained by my byte construction, which is a second reason to readpackB_blocked/write_container's row walk rather than infer it: with 6144 columns (5120 padded to a 256 multiple) the block offsets differ from the 68-byte stride I crafted.Next step is mechanical once that walk is read: decode the row the writer would place the first real row at, assert −10.0, then run the mutation the issue asks for (swap two rows' scales in the writer; arm 2 must fail). I would rather hand over a measured localisation than a test that is green for the wrong reason.
bong-water-water-bong commentedon Nov 10, 2026
Two more measured facts, both of which change what the read-back has to do (local commit
+1onnpu-pack-external-success).1. The stored shape is
[tile count, bytes per tile], not[rows, cols].pack_import.cppwritesw.add(prefix, "I8", {nt, kImportTileBytes}, …)withkImportTileBytes = q4nx::Chunk::Q4_1F = 6144. So my q_proj's19,660,800 Bis 3200 tiles × 6144 B, and "shape 3200 × 6144" was that pair — which explains the 0.0 / 26.0 mixture I reported earlier: I was indexing rows against a 3200-row grid that does not exist.2. The read-back has to take the device loader route, and it did not fire.
q4nx_dequant_tile_row()decodes only the three 5120/Q4K/Q8 chunk kinds and refuses 6144 outright (−1).q4nx_device.cpp:263-266handles this by repacking first:as_q4_1(x)then decode 5120-B tiles. I calledq4nx::as_q4_1on{I8, shape {5120,5120}, bytes 19660800}and got 19,660,800 B back unchanged — a pass-through branch rather than the 3200 × 5120 = 16,384,000 B repack.So the next step is narrow and mechanical: read what
as_q4_1(npu/q4nx.cpp:210) dispatches on — the caller has to satisfy its precondition for the conversion to happen — then assert −10.0 on the first real row and run the mutation. The fixture, the import, the geometry and the crafted blob are all in place and green; only the tile-read-back chain is unresolved. Still not pushed, for the same reason as before.bong-water-water-bong commentedon Nov 10, 2026
The read-back chain works end to end and the suite is green (local commit
01c90006, still not pushed). Three findings, the middle one a correction of my own.1.
chunk_of()readsshape.back()— which is why the writer stores{ntiles, chunk_bytes}. Giving the Tensor{rows, cols}madechunk_of()see 5120 (Q4_1) andas_q4_1()copied the bytes straight through; giving it{3200, 6144}made it repack 19,660,800 → 16,384,000 B (3200 tiles) and the rows decode. So the stored shape is not an accident, it is the chunk tag.2. The value is 22.0, not the −10.0 I asserted — and my first derivation was wrong, not the writer. MatMulNBits codes are unsigned, with the zero point shifting them:
w = (nibble − zp) · scale = (12 − 1) · 2.0 = 22.0.pack_import.cppsays so itself (w = (code_nibble - zp_nibble) * scale). I had read nibble 12 as int4 −4; every row decoding to a clean 22.0 is what exposed it.3. The mutation is caught by the suite but not by my arm. Doubling one scale slot in
container_from_external_q4fails the suite (exit 1):but my "every row decodes to 22.0" arm stays green — so it is less sensitive than the sibling arms on this writer, and I have not explained why the doubled slot does not reach the q_proj values it decodes. I made the arm scan every row (a row-0-only check is blind to a wrong slot anywhere else) and it still does not fire.
That is why this is still local: the issue asks that arm 2 fail on the mutation, and it does not. What the work does establish is that the fixture, the satisfying geometry, the import, the container and a value-level read-back all exist and are green — so the remaining question is narrowly about where the doubled slot is lost, not about whether a success-path test is possible.
bong-water-water-bong commentedon Nov 10, 2026
Done — PR #1902. The mutation now bites the arm, and the mechanism explains why my first version missed it.
The mutation (double one scale slot in
container_from_external_q4) fails it as:The reason my q-only arm stayed green:
container_from_external_q4()runs on the FUSED qkv, soscale[size()/2]of a 15360-row pack lands ink_projrow 2560 — exactly half of 5120 — and q_proj is untouched. The arm now checks q, k and v, which is where the writer actually works.Also settled since the last comment: the value is 22.0 (
(nibble − zp)·scale, MatMulNBits codes are unsigned —pack_import.cppsays so in its own comment), and the read-back is the device loader's route (chunk_of()readsshape.back(), so the Tensor is shaped{ntiles, chunk_bytes};as_q4_1()repacks 19,660,800 → 16,384,000 B and the rows decode). No second decoder was written for the test.Whole suite green with the writer intact;
npu/forward/src/q4nx_dequant.cis added to the existingnpu_pack_import_testtarget, so no new test target and no workflow scope.bong-water-water-bong commentedon Nov 10, 2026
Two PRs add the missing
import_external()success path — map, and a decisionBoth PRs exist because of the same gap, and both describe it identically in substance: every existing
import_pack()arm on the 1.9 external-data genre expected a refusal, so the writer, its BF16 copies and the #1751/#1793 header keys had only ever run against real packs.check(arms7eaa7a0N*(K/128+1)*68),why/refusedempty andmodel.q4nxexists,num_attention_heads/num_key_value_headsmatch,q4nx_weight_layoutisb8where expected and absent where a default must not emit it,tokenizer.jsoncopied, per-tensor(hi−lo) == RT·KT·tb, plus the newer int4 sign-extend reference (all 16 nibbles, `8c241f3model.q4nxexists, the container carries the expected role files, and abad == 0countRecommendation: land #1863, close #1902 as covered. #1863 asserts the writer's contract (the sizes and keys a reader must find), while #1902 asserts that the path runs and produces a container — a smoke test. Both are good, but only one can land in this test, and the one with ten assertions subsumes the other's four.
What #1902 has that #1863 should borrow before #1902 closes: the role-file assertion (
the container carries <role>) and the aggregatebad == 0count. #1863 checks the token table and the config, but a multi-component genre's role files are exactly what a future multi-component pack will depend on — one more assertion in #1863 and nothing from #1902 is lost.And one thing to keep visible when they close: this is the third cluster tonight where the same gap drew two or more PRs (#1889 → #1895/#1896/#1897; #1864 → #1891/#1898/#1866; this). The pattern is that a CHANGES-driven requirement is discovered by several lanes at once from the same audit row — worth the lanes checking
gh pr list --searchfor the issue number before starting, which is the rule the brief already states formain-is-red fixes.