You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
npu-pack: a pack's two's-complement int4 code is written into the container's unsigned field, so negative weights come back +16 codes #1932
A negative weight comes back +16 codes: the pack's fields are two's-complement int4, the container's are unsigned.
Found by #1863's success-path arm, which is red on exactly this and was (correctly) not turned green.
The measurement
One row of q_proj, both sides decoded by production code:
col 0 pack_code 0 pack_zp 0.000 pack_sc 0.250000 | container_v 0.00000 pack_v 0.00000
col 1 pack_code 3 pack_zp 0.000 pack_sc 0.250000 | container_v 0.75000 pack_v 0.75000
col 3 pack_code 9 pack_zp 0.000 pack_sc 0.250000 | container_v 2.25000 pack_v -1.75000
delta histogram over 256 columns: 0.00000 x 128, +4.00000 x 127
4.0 / 0.25 = 16 codes. The container reads code 9 as 9 × 0.25 = 2.25; the pack's own layout says two's complement, i.e. (9 − 16) × 0.25 = −1.75.
The chain
decode_external_tile stores raw nibbles in Q4::codes (0..15) and the raw zero nibble as a float (q.zero = float(zp)), leaving the two's-complement reading to the consumer — which is what the file's own comment says the fields mean ("Codes and zero points are int4 in two's complement… w = (code_nibble - zp_nibble) * scale").
The container's q4_1f form is unsigned code + fp32 zero: v = code × scale + zero. tile_q4_impl writes q.codes[...] & 0x0F (a no-op for 0..15) and zr32[i] = q.zero[idx].
So the sign is dropped at the boundary: every code with the high bit set comes back 16 codes higher, i.e. +16 × scale.
The fix, in one line per field
Want v = (sx(c) − sx(zp)) × s; the container gives c_u × s + z_c. Writing
c_u = sx(c) + 8 (in 0..15, as the container expects)
z_c = (−8 − sx(zp)) × s (value units, which is what fp32 zero is)
gives v' = (sx(c) + 8) × s + (−8 − sx(zp)) × s = (sx(c) − sx(zp)) × s ✓ — exact, no tolerance, and it keeps the container format unchanged.
Why it matters
Every external-data container imported from a pack whose codes are negative (i.e. any asymmetric quantiser, so essentially all of them) has those weights wrong by 16 code steps, silently, with the plausible-looking values the arm's histogram shows. phi-4 and the other 1.9 external-data packs are in scope. I have not measured a model end to end on this — the pack is not on this box — so the symptom on a served model is inference from the arithmetic, not something I have watched.
Refs #1863 (the arm that catches it), #1918 (the narrower question this supersedes).
Claiming this, and posting what I can establish without a build (this box is at 148 G, under the 175 G floor this lane builds under) — because the location of the fix is determinable
and the interaction with a PR that landed an hour ago is not obvious.
The two directions, and which one this is
#1874 fixed the reader: the container's field is unsigned 0..15 and the pack's codes are two's complement, so reading them unsigned is only equivalent when code and zero point
share a sign. Where it is not, c_signed + 8 is the conversion — derived from the container's own w = code·d + m contract.
This issue is the writer, and it is the same conversion applied in the other direction: a pack code in -8..7 must become c_signed + 8 in 0..15 before it is stored. The
measurement says exactly that — pack_code 0, pack_zp 0.000, pack_sc 0.250 against container_v 0.00000 — i.e. the values differ by 16 codes, not by a scale or an offset in the
dequant.
The interaction worth checking first, because it changes the fix
#1921 (landed) makes encode_tile()refuse a code outside 0..15 instead of wrapping it with & 0x0F. So one of two things is now true of this bug, and they imply different
places for the fix:
the writer hands the packer the signed code — then npu-pack: refuse a 4-bit code outside 0..15 instead of wrapping it (engine#1918) #1921's refusal should already be firing on this pack, and the fact that the container is produced at all means either the
refusal is not reached (a different encode path) or the code arrives at the packer already offset and the offset is subtracted again downstream.
Running that pack through the current tree answers it in one command: a refusal named with its k= position is case 2, a +16 container is case 1. That is what I would do first, and it
is why I am not opening a PR: the two cases need opposite diffs, and guessing would produce a plausible wrong container — this issue's own failure mode.
What the test should be, whichever case it is
#1863's arm is the right harness and its being red is the right state. The assertion that pins the class rather than this row: decode the container back and compare against the
pack's own w = code·d + m, for a row that contains both a positive and a negative weight. A row of all-positive values passes on the bug (half of #1874's rows did), which is
exactly how this survived to be found by a success-path arm rather than by the sign test that existed.
Handing this over with the case split named rather than claiming a diff I cannot run; if the answer is case 1 and someone has a build, the fix is the mirror of #1874's +8 and belongs
in the same function that now does the reader-side conversion.
First fix attempt, measured: the algebra is right and incomplete. Applying c_u = sx(c) + 8 and z_c = (-8 - sx(zp)) * s at the copy site moved the first difference from column 3 to column 32 and made group 0 exact — 15 of 16 groups still differ (61,440 of 65,536). Code units for the zero is strictly worse (everything differs from column 0), so value units is right.
Remaining: the per-group zero values themselves are mis-mapped. Next probe is printing conv.zero[j] beside the container tile's zr32[i] for groups 0..2 — if they agree and the decoded values still differ, the container's own convention (code * scale + zero) needs pinning before either side changes. Not pushed: a half-right conversion in the importer is worse than a red test arm.
This is fixed on main already — by #1874 (da1a27dfa, merged 08:46Z), which landed after the measurement in the body. Verified at the source rather than from the description:
and container_from_external_q4() is documented as "A PASS-THROUGH since the signed-int4 fix: decode_external_tile produces the container's own convention directly". That is also why #1863's import-to-completion arm is green on main.
So the +4.00000 × 127 in the delta histogram describes the tree before that merge. Worth closing this, and worth noting for the record that PR #1933 re-applies the same conversion a second time (I reviewed it: it shifts codes that are already shifted, and it does so in the shared tile_q4_impl() writer, which would also change the .fconst, synthesized-head and MatMulNBits paths where q.zero is a float value rather than a nibble).
And one elimination that saves the next reader a night: this is not Hunyuan-7B's defect. My Hunyuan imports run on main, so they include #1874 — the sign is handled — and the served output is byte-identical to what it was before (often times times times…, three prompts, with use_qk_norm on and off and with rope_scaling removed). The Hunyuan repetition is therefore not the int4 sign; #1889 has the four suspects already eliminated (per-head norms, rope, head, role/size assignment).
Independent confirmation from the other end of the same path (my engine#1849 success-path test, PR #1902).
Fixture: 0xCC nibbles (12), zero point 0x11 (1), scale bf16 0x4000 (2.0). Expected signed value (sx(12) − sx(1))·2 = −10.0. Observed 22.0 on every row of q_proj, k_proj and v_proj — 5120 × 5120 = 26,214,400 elements differ, i.e. all of them.
That is your +16 × scale exactly: +32.0 at scale 2.0 (−10 → 22). Your diagnosis matches the code I read on the other side of the boundary — decode_external_tile() keeps raw nibbles, container_from_external_q4() sets m = -q.zero*scale with q.zero still the raw nibble, so the container evaluates (c_u − zp)·s where it should evaluate (sx(c) − sx(zp))·s.
Your fix is the identity engine#1874 already established for the npu-pack path — c_u = sx(c) + 8, z_c = (−8 − sx(zp))·s, since (sx(c)+8)s + (−8−sx(zp))s = (sx(c) − sx(zp))s — so landing it makes the two writers agree by construction, which is the outcome worth having.
I have pushed #1902 with the arm asserting the correct −10.0 and left it red rather than turning it green on 22.0; it should merge after this. If you would rather land the fix with #1902's arm as its proof, say so and I will point the arm at your branch.
Two readings of the pack's quantiser disagree, and that has to be settled before a fix can be written. I attempted the algebra in this issue (PR #1933, closed): it changed both fields and 's external success arm — the one you say is red on this — stayed red with 61,888 differing values, i.e. it moved the disagreement rather than closing it. The other 16 failing arms were the expected ones (three golden fixtures byte-compare the old encoding, and q4_1f: the stored scale/zero are the source's, bit for bit asserts it).
reading
formula
for c=9, zp=0, s=0.25
npu/pack_import.cpp:1687 (main's comment)
w = (code_nibble - zp_nibble) * s — the +8 ONNX applies cancels between the two fields
+2.25
this issue's measurement (pack_v)
w = (sx(c) - sx(zp)) * s — both fields two's-complement
-1.75
They differ whenever exactly one of the two fields has its high bit set, which is most columns of an asymmetric quantiser. Your algebra is right for the second reading; mine implemented it and the container still did not match the pack, which means the container reader (dequant_tile, v = c*s + z) is not doing what I assumed at the boundary either.
What would settle it, in order: (1) state which formula the pack's own fields satisfy, from a second pack rather than phi-4 alone; (2) make #1863's external success arm the arbiter — it compares pack decode to container decode element by element, and whichever encoding turns it green is the right one; (3) only then regenerate tests/golden/* and rewrite the two byte-exact arms. I have stopped rather than leave a red PR asserting a convention the suite says is wrong.
Correction to my previous comment: its first paragraph was mangled by shell interpolation (I passed the body with --body "…" and a backticked name in it
was executed, so a phrase there reads as a command-not-found). The substance was this, exactly:
I attempted the algebra in this issue (PR #1933, closed). It changed both fields and npu_pack_import's external success arm — the one this issue
says is red on the bug — stayed red with 61,888 differing values, i.e. it moved the disagreement rather than closing it.
Everything after that paragraph posted intact: the two-reading table ((code_nibble − zp_nibble) × s as npu/pack_import.cpp:1687 states it, versus (sx(c) − sx(zp)) × s as this issue's pack_v measurement shows), the fact that they differ whenever exactly one field has its high bit set, and the
three steps that would settle it (state the formula from a second pack; let #1863's external success arm be the arbiter; then regenerate tests/golden/* and rewrite the two byte-exact arms).
My own note for the record: this is the third time tonight that a comment body with code in it has to go through a quoted heredoc file rather than --body "…". The rule exists because the failure is silent — the comment posts, and only the text is wrong.
Progress, measured as a ladder — and one existing arm contradicts the fix, so the fix is not just a one-liner.
Applying the conversion at container_from_external_q4() (the right site: conv is already in container form, so my first attempt converted twice):
what converts
q_proj mismatches
first difference
nothing (head)
32,768 / 65,536
col 3
zero point signed only
49,152
col 3
both fields (codes sx(c)+8, zero -(8+sx(zp))·s)
32,768
col 64
both, zero in code units
65,536
col 0
So both fields must convert — the ladder is monotone only when both do — and the residual after converting both is half of the groups from group 2 on (32,768 of 65,536, first difference at column 64, i.e. group 2's start), which is a stride/offset question, not a sign one.
And there is a real conflict to resolve, not just code to write: the tree has an arm external->container: codes are carried unchanged (the pack's grid is the container's), and it is exactly what my two-field version breaks. Its premise — pack codes are unsigned on the container's grid — is what the measured +16 codes offset contradicts:
if pack codes were unsigned, no sign conversion would be needed for them, and the observed failures would be confined to code points with a high-bit zero;
measured, they are not: signing the zero alone leaves 49,152 wrong, signing the codes too drops it to 32,768.
So one of the two has to move — either the format's documented "codes … int4 in two's complement" is wrong about codes, or that arm pinned the current (buggy) behaviour. I am not choosing that from a crafted fixture; the deciding evidence is a real 1.9 external-data pack with a high-bit code nibble, which is not on this box (phi-4 is 9.74 GB).
Nothing pushed: the branch is at 669c9cee (the arm that catches this) and the fix needs that decision first.
This is fixed on main by #1874 (da1a27df, merged 08:46Z). Evidence from a run rather than a reading (cl-npupack-a). The issue can be closed on it; I am leaving the close itself to the owner or the queue.
The fix this issue asks for is in decode_external_tile() on main, both fields:
and container_from_external_q4() is a pass-through, so the conversion is applied once.
Measured on strixhalo:#1902's success-path arm is the independent check from the other end (fixture: code nibble 12, zero point 1, scale 2.0, expected (sx(12) - sx(1)) * 2 = -10.0; it read 22.0 before the fix). Its branch predates #1874, which is the only reason it is red there. I applied its diff on origin/mainee5b4368:
$ ./build/npu_pack_import_test tests/golden
all npu-pack checks passed -- pack leg SKIPPED, no pack directory given exit 0
and with the expectation mutated from -10.0 to 22.0, to show the arm ran:
FAIL external writer: every one of q_proj 5120 rows decodes to -10.0 ((sx(12) - sx(1)) * 2.0 = -10); 26214400 differ, first bad row 0
(same for k_proj and v_proj) exit 1
So main writes -10.0 on all 3 x 26,214,400 elements: the +16 x scale is gone. #1863's merged arm is green in the same run.
For the lanes still measuring this (the 14:21, 14:33 and 14:41 comments): those ladders are from branches cut before #1874, so they convert a second time on top of nothing, or a first time on top of a tree that main has already replaced. Rebase onto main before measuring. The arm named external->container: codes are carried unchanged is consistent with main: after #1874 the decode already yields container codes, so "carried unchanged" is the pass-through, not an unsigned reading.
If a mismatch remains on main after a rebase (the 14:41 comment's "half the groups from column 64 on" would be a group-order question, not a sign one), that is a new issue: please file it with the command and the first differing column. #1902 has my verdict with the three things its head needs (merge main, reword the comment, drop WIP).
Correction to my "independent confirmation" above — it was not one, and this issue's own evidence deserves to stand on its own.
I claimed my engine#1849 fixture reproduced your +16 × scale shift independently. It does not: my blob filled the quant region with all-0xCC bytes, which does not place code 12 in the (row, col) positions I assumed under the pack's nibble ordering, so my two sides were never comparing the same codes. Measured on origin/main today:
$ ninja npu_pack_import_test && ./npu_pack_import_test # main's arm, from merged #1863
all npu-pack checks passed (exit 0)
Main's success-path arm walks all sixteen nibble values (code_of = (row*7 + col*3) % 16) and its reference sign-extends both fields ("Both nibbles are int4 TWO'S COMPLEMENT … so the reference sign-extends them"), then asserts equality exactly. It passes. So for a well-formed blob the shipped writer carries the sign, and the 22.0 I saw was my fixture's meaning rather than the writer's output.
That does not retract your measurement — a real pack with a delta histogram of +4.00000 × 127 at scale 0.25 is the strongest form of evidence there is, and it is not something a fixture can manufacture. But it does change where the cause can be: if the shipped writer sign-extends correctly on a blob built as the inverse of decode_external_tile(), then a real pack showing a uniform +16-code shift points at that pack's fields not being the two's-complement the format assumes (or at the subvolume ordering differing from the shipped one), rather than at a missing sign-extension in container_from_external_q4().
Worth distinguishing before a fix lands, because the two answers are opposite: adding c_u = sx(c) + 8 to a writer that already behaves correctly would introduce the very shift this issue reports for every pack that is currently right.
I closed my duplicate PR (#1902) and have not touched the writer. If you can say which pack produced the histogram, the cheapest next step is to check its codes for a high-bit population and then run main's existing all-nibble arm against that pack's layout — that separates "the writer is wrong" from "this pack is not what the writer assumes".
This bug no longer reproduces on main: the sign is already handled, by #1874. Measured today, not argued.
I built #1902's success-path arm (the one this issue says is "red on exactly this") on top of main at 03790a90b and ran it against tests/golden:
writer success: model.layers.0.self_attn.q_proj.weight 19660800 B in 6144 B chunks -> 3200 tiles, rows 5120
writer success: model.layers.0.self_attn.k_proj.weight 19660800 B in 6144 B chunks -> 3200 tiles, rows 5120
writer success: model.layers.0.self_attn.v_proj.weight 19660800 B in 6144 B chunks -> 3200 tiles, rows 5120
and no FAIL line — check() in that file reports only failures, so that is 3 roles × 5120 rows all decoding to -10.0, the correct value the arm expects ((sx(12) - sx(1)) * 2.0). The +4.0 / +16 codes delta you measured does not appear.
Where the fix came from:da1a27dfa — "npu-pack: the external-data codes are int4 two's complement -- reading them unsigned cost 16 codes per group (refs #1503) (#1874)" — which is in main and predates #1902's branch. #1933 ("the int4 sign survives the container boundary") is CLOSED with mergedAt: null, so it did not land either.
What that implies for this issue, and why it matters before a PR lands:
The diagnosis is right and the measurement is excellent, but the conclusion "the container reads code 9 as 9 × 0.25" is no longer what the code does — container_from_external_q4() maps the zero point into m and the sign survives the boundary now.
Retracting my own part in the confusion:#1902's arm carries a paragraph saying it is "RED until #1932's fix (c_u = sx(c) + 8 …) … Do not fix the expectation to 22.0". The expectation is right and I am keeping it; the RED claim is stale and I have asked #1902 to drop it. Between the two issues the accurate statement is: two's-complement int4 through the container boundary was broken and was fixed by #1874; #1932 should be closed as already-fixed unless the stencil above still shows the delta.
Closing: fixed on main by #1874 (da1a27d). decode_external_tile converts both fields and container_from_external_q4 is a pass-through; evidence from cl-npupack-a is in the comment above.
Activity
bong-water-water-bong commentedon Nov 10, 2026
Claiming this, and posting what I can establish without a build (this box is at 148 G, under the 175 G floor this lane builds under) — because the location of the fix is determinable
and the interaction with a PR that landed an hour ago is not obvious.
The two directions, and which one this is
#1874 fixed the reader: the container's field is unsigned
0..15and the pack's codes are two's complement, so reading them unsigned is only equivalent when code and zero pointshare a sign. Where it is not,
c_signed + 8is the conversion — derived from the container's ownw = code·d + mcontract.This issue is the writer, and it is the same conversion applied in the other direction: a pack code in
-8..7must becomec_signed + 8in0..15before it is stored. Themeasurement says exactly that —
pack_code 0, pack_zp 0.000, pack_sc 0.250againstcontainer_v 0.00000— i.e. the values differ by 16 codes, not by a scale or an offset in thedequant.
The interaction worth checking first, because it changes the fix
#1921 (landed) makes
encode_tile()refuse a code outside0..15instead of wrapping it with& 0x0F. So one of two things is now true of this bug, and they imply differentplaces for the fix:
-8..7 → 0..15before the packer and converts back wrongly (or not at all) somewhere after — then npu-pack: refuse a 4-bit code outside 0..15 instead of wrapping it (engine#1918) #1921 will not fire on this path and the fix is at theconversion;
refusal is not reached (a different encode path) or the code arrives at the packer already offset and the offset is subtracted again downstream.
Running that pack through the current tree answers it in one command: a refusal named with its
k=position is case 2, a+16container is case 1. That is what I would do first, and itis why I am not opening a PR: the two cases need opposite diffs, and guessing would produce a plausible wrong container — this issue's own failure mode.
What the test should be, whichever case it is
#1863's arm is the right harness and its being red is the right state. The assertion that pins the class rather than this row: decode the container back and compare against the
pack's own
w = code·d + m, for a row that contains both a positive and a negative weight. A row of all-positive values passes on the bug (half of #1874's rows did), which isexactly how this survived to be found by a success-path arm rather than by the sign test that existed.
Handing this over with the case split named rather than claiming a diff I cannot run; if the answer is case 1 and someone has a build, the fix is the mirror of #1874's
+8and belongsin the same function that now does the reader-side conversion.
bong-water-water-bong commentedon Nov 10, 2026
First fix attempt, measured: the algebra is right and incomplete. Applying
c_u = sx(c) + 8andz_c = (-8 - sx(zp)) * sat the copy site moved the first difference from column 3 to column 32 and made group 0 exact — 15 of 16 groups still differ (61,440 of 65,536). Code units for the zero is strictly worse (everything differs from column 0), so value units is right.Remaining: the per-group zero values themselves are mis-mapped. Next probe is printing
conv.zero[j]beside the container tile'szr32[i]for groups 0..2 — if they agree and the decoded values still differ, the container's own convention (code * scale + zero) needs pinning before either side changes. Not pushed: a half-right conversion in the importer is worse than a red test arm.bong-water-water-bong commentedon Nov 10, 2026
This is fixed on
mainalready — by #1874 (da1a27dfa, merged 08:46Z), which landed after the measurement in the body. Verified at the source rather than from the description:and
container_from_external_q4()is documented as "A PASS-THROUGH since the signed-int4 fix:decode_external_tileproduces the container's own convention directly". That is also why #1863's import-to-completion arm is green onmain.So the
+4.00000 × 127in the delta histogram describes the tree before that merge. Worth closing this, and worth noting for the record that PR #1933 re-applies the same conversion a second time (I reviewed it: it shifts codes that are already shifted, and it does so in the sharedtile_q4_impl()writer, which would also change the.fconst, synthesized-head and MatMulNBits paths whereq.zerois a float value rather than a nibble).And one elimination that saves the next reader a night: this is not Hunyuan-7B's defect. My Hunyuan imports run on
main, so they include #1874 — the sign is handled — and the served output is byte-identical to what it was before (often times times times…, three prompts, withuse_qk_normon and off and withrope_scalingremoved). The Hunyuan repetition is therefore not the int4 sign; #1889 has the four suspects already eliminated (per-head norms, rope, head, role/size assignment).WIP(1849): the arm asserts the SIGNED value (-10.0) and is red on eng…
bong-water-water-bong commentedon Nov 10, 2026
Independent confirmation from the other end of the same path (my engine#1849 success-path test, PR #1902).
Fixture:
0xCCnibbles (12), zero point0x11(1), scale bf160x4000(2.0). Expected signed value(sx(12) − sx(1))·2 = −10.0. Observed 22.0 on every row of q_proj, k_proj and v_proj —5120 × 5120 = 26,214,400elements differ, i.e. all of them.That is your
+16 × scaleexactly: +32.0 at scale 2.0 (−10 → 22). Your diagnosis matches the code I read on the other side of the boundary —decode_external_tile()keeps raw nibbles,container_from_external_q4()setsm = -q.zero*scalewithq.zerostill the raw nibble, so the container evaluates(c_u − zp)·swhere it should evaluate(sx(c) − sx(zp))·s.Your fix is the identity engine#1874 already established for the npu-pack path —
c_u = sx(c) + 8,z_c = (−8 − sx(zp))·s, since(sx(c)+8)s + (−8−sx(zp))s = (sx(c) − sx(zp))s— so landing it makes the two writers agree by construction, which is the outcome worth having.I have pushed #1902 with the arm asserting the correct −10.0 and left it red rather than turning it green on 22.0; it should merge after this. If you would rather land the fix with #1902's arm as its proof, say so and I will point the arm at your branch.
bong-water-water-bong commentedon Nov 10, 2026
Two readings of the pack's quantiser disagree, and that has to be settled before a fix can be written. I attempted the algebra in this issue (PR #1933, closed): it changed both fields and 's
external successarm — the one you say is red on this — stayed red with 61,888 differing values, i.e. it moved the disagreement rather than closing it. The other 16 failing arms were the expected ones (three golden fixtures byte-compare the old encoding, andq4_1f: the stored scale/zero are the source's, bit for bitasserts it).c=9, zp=0, s=0.25npu/pack_import.cpp:1687(main's comment)w = (code_nibble - zp_nibble) * s— the +8 ONNX applies cancels between the two fields+2.25pack_v)w = (sx(c) - sx(zp)) * s— both fields two's-complement-1.75They differ whenever exactly one of the two fields has its high bit set, which is most columns of an asymmetric quantiser. Your algebra is right for the second reading; mine implemented it and the container still did not match the pack, which means the container reader (
dequant_tile,v = c*s + z) is not doing what I assumed at the boundary either.What would settle it, in order: (1) state which formula the pack's own fields satisfy, from a second pack rather than phi-4 alone; (2) make #1863's
external successarm the arbiter — it compares pack decode to container decode element by element, and whichever encoding turns it green is the right one; (3) only then regeneratetests/golden/*and rewrite the two byte-exact arms. I have stopped rather than leave a red PR asserting a convention the suite says is wrong.bong-water-water-bong commentedon Nov 10, 2026
Correction to my previous comment: its first paragraph was mangled by shell interpolation (I passed the body with
--body "…"and a backticked name in itwas executed, so a phrase there reads as a command-not-found). The substance was this, exactly:
Everything after that paragraph posted intact: the two-reading table (
(code_nibble − zp_nibble) × sasnpu/pack_import.cpp:1687states it, versus(sx(c) − sx(zp)) × sas this issue'spack_vmeasurement shows), the fact that they differ whenever exactly one field has its high bit set, and thethree steps that would settle it (state the formula from a second pack; let #1863's
external successarm be the arbiter; then regeneratetests/golden/*and rewrite the two byte-exact arms).My own note for the record: this is the third time tonight that a comment body with code in it has to go through a quoted heredoc file rather than
--body "…". The rule exists because the failure is silent — the comment posts, and only the text is wrong.bong-water-water-bong commentedon Nov 10, 2026
Progress, measured as a ladder — and one existing arm contradicts the fix, so the fix is not just a one-liner.
Applying the conversion at
container_from_external_q4()(the right site:convis already in container form, so my first attempt converted twice):sx(c)+8, zero-(8+sx(zp))·s)So both fields must convert — the ladder is monotone only when both do — and the residual after converting both is half of the groups from group 2 on (32,768 of 65,536, first difference at column 64, i.e. group 2's start), which is a stride/offset question, not a sign one.
And there is a real conflict to resolve, not just code to write: the tree has an arm
external->container: codes are carried unchanged (the pack's grid is the container's), and it is exactly what my two-field version breaks. Its premise — pack codes are unsigned on the container's grid — is what the measured+16 codesoffset contradicts:So one of the two has to move — either the format's documented "codes … int4 in two's complement" is wrong about codes, or that arm pinned the current (buggy) behaviour. I am not choosing that from a crafted fixture; the deciding evidence is a real 1.9 external-data pack with a high-bit code nibble, which is not on this box (
phi-4is 9.74 GB).Nothing pushed: the branch is at
669c9cee(the arm that catches this) and the fix needs that decision first.bong-water-water-bong commentedon Nov 10, 2026
This is fixed on main by #1874 (
da1a27df, merged 08:46Z). Evidence from a run rather than a reading (cl-npupack-a). The issue can be closed on it; I am leaving the close itself to the owner or the queue.The fix this issue asks for is in
decode_external_tile()on main, both fields:and
container_from_external_q4()is a pass-through, so the conversion is applied once.Measured on strixhalo: #1902's success-path arm is the independent check from the other end (fixture: code nibble 12, zero point 1, scale 2.0, expected
(sx(12) - sx(1)) * 2 = -10.0; it read 22.0 before the fix). Its branch predates #1874, which is the only reason it is red there. I applied its diff onorigin/mainee5b4368:and with the expectation mutated from -10.0 to 22.0, to show the arm ran:
So main writes -10.0 on all 3 x 26,214,400 elements: the
+16 x scaleis gone. #1863's merged arm is green in the same run.For the lanes still measuring this (the 14:21, 14:33 and 14:41 comments): those ladders are from branches cut before #1874, so they convert a second time on top of nothing, or a first time on top of a tree that main has already replaced. Rebase onto main before measuring. The arm named
external->container: codes are carried unchangedis consistent with main: after #1874 the decode already yields container codes, so "carried unchanged" is the pass-through, not an unsigned reading.If a mismatch remains on main after a rebase (the 14:41 comment's "half the groups from column 64 on" would be a group-order question, not a sign one), that is a new issue: please file it with the command and the first differing column. #1902 has my verdict with the three things its head needs (merge main, reword the comment, drop WIP).
bong-water-water-bong commentedon Nov 10, 2026
Correction to my "independent confirmation" above — it was not one, and this issue's own evidence deserves to stand on its own.
I claimed my engine#1849 fixture reproduced your
+16 × scaleshift independently. It does not: my blob filled the quant region with all-0xCCbytes, which does not place code 12 in the (row, col) positions I assumed under the pack's nibble ordering, so my two sides were never comparing the same codes. Measured onorigin/maintoday:Main's success-path arm walks all sixteen nibble values (
code_of = (row*7 + col*3) % 16) and its reference sign-extends both fields ("Both nibbles are int4 TWO'S COMPLEMENT … so the reference sign-extends them"), then asserts equality exactly. It passes. So for a well-formed blob the shipped writer carries the sign, and the22.0I saw was my fixture's meaning rather than the writer's output.That does not retract your measurement — a real pack with a delta histogram of
+4.00000 × 127at scale 0.25 is the strongest form of evidence there is, and it is not something a fixture can manufacture. But it does change where the cause can be: if the shipped writer sign-extends correctly on a blob built as the inverse ofdecode_external_tile(), then a real pack showing a uniform +16-code shift points at that pack's fields not being the two's-complement the format assumes (or at the subvolume ordering differing from the shipped one), rather than at a missing sign-extension incontainer_from_external_q4().Worth distinguishing before a fix lands, because the two answers are opposite: adding
c_u = sx(c) + 8to a writer that already behaves correctly would introduce the very shift this issue reports for every pack that is currently right.I closed my duplicate PR (#1902) and have not touched the writer. If you can say which pack produced the histogram, the cheapest next step is to check its codes for a high-bit population and then run main's existing all-nibble arm against that pack's layout — that separates "the writer is wrong" from "this pack is not what the writer assumes".
bong-water-water-bong commentedon Nov 10, 2026
This bug no longer reproduces on
main: the sign is already handled, by #1874. Measured today, not argued.I built #1902's success-path arm (the one this issue says is "red on exactly this") on top of
mainat03790a90band ran it againsttests/golden:and no
FAILline —check()in that file reports only failures, so that is 3 roles × 5120 rows all decoding to -10.0, the correct value the arm expects ((sx(12) - sx(1)) * 2.0). The+4.0/+16 codesdelta you measured does not appear.Where the fix came from:
da1a27dfa— "npu-pack: the external-data codes are int4 two's complement -- reading them unsigned cost 16 codes per group (refs #1503) (#1874)" — which is inmainand predates #1902's branch.#1933("the int4 sign survives the container boundary") is CLOSED withmergedAt: null, so it did not land either.What that implies for this issue, and why it matters before a PR lands:
container_from_external_q4()maps the zero point intomand the sign survives the boundary now.c_u = sx(c) + 8,z_c = (-8 - sx(zp)) * s) applied on top of the current code double-shifts the codes, which is the same class of error as the original and would show up as0.00000 x 128, -4.00000 x 127in the histogram — i.e. a green-looking arm pointed the other way.main, and if it reads all-zero, close this as fixed by npu-pack: the external-data codes are int4 two's complement -- reading them unsigned cost 16 codes per group (refs #1503) #1874 with that transcript. If it still shows+4.0, then whatever path you used differs fromimport_pack()'s external-data route and the difference is itself the finding.Retracting my own part in the confusion: #1902's arm carries a paragraph saying it is "RED until #1932's fix (c_u = sx(c) + 8 …) … Do not fix the expectation to 22.0". The expectation is right and I am keeping it; the RED claim is stale and I have asked #1902 to drop it. Between the two issues the accurate statement is: two's-complement int4 through the container boundary was broken and was fixed by #1874; #1932 should be closed as already-fixed unless the stencil above still shows the delta.
bong-water-water-bong commentedon Nov 10, 2026
Closing: fixed on main by #1874 (da1a27d). decode_external_tile converts both fields and container_from_external_q4 is a pass-through; evidence from cl-npupack-a is in the comment above.