You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
npu-pack: locate_fconst_with_header() rewrites every 1.9 pack's header list to the 2-header shape, so Qwen3's 4-header blocks read the k_norm header as post-attention (#1819) #1831
locate_fconst_with_header() (merged in #1819, dae560b) overwrites the per-block header list for every 1.9 pack that has a 1.9 header, including when locate_fconst()'s epsilon vote already found the layout, and it always writes the 2-header shape:
This runs after g.stride = stride8 with no check on whether g.hdr was already filled. locate_fconst() records period − 1 inner headers (for (int i = 1; i < g.period; ++i) g.hdr.push_back(...)), and the readers choose the block layout from that list:
plan() branches on hdr.size() == 1 (2-header: Qwen2.5/Llama) versus hdr.size() == 3 (4-header: Qwen3, q/k/v, then k_norm and q_norm, then o_proj);
FconstGeom::post_attn_hdr() returns hdr.back(), which is the post-attention norm and the start of the MLP section.
So for a Qwen3 1.9 pack, hdr goes from 3 entries to 1 and the block is read as 2-header. post_attn_hdr() then returns the k_norm section's header, which is exactly the misread FconstGeom's own comment records as engine#1003 ("imported the k_norm section's padding as post_attention_layernorm on every Qwen3 pack"). TNPU-ALL's scoreboard lists Qwen3-1.7B, Qwen3-4B and Qwen3-4B-Instruct-2507 among the 1.9 packs that run today.
When the vote does fail, the same block also picks the smallest repeating residue (if (n > best) keeps the first maximum). In a 4-header block the three inner residues repeat equally often, so it picks the k_norm header and calls it h1 of a 2-header block.
Not yet measured on a pack. No Qwen3 1.9 pack is on the review box, and npu_pack_import_test's pack leg is the 2-header 0.5B, which is why nothing went red. Measuring it: import any Qwen3 1.9 pack on main and on dae560b^, and diff the post_attention_layernorm tensors (or check the .fconst plan's header count).
Fix:
Keep locate_fconst()'s list when it filled one.
When filling it from residues, keep every residue that repeats: one is the 2-header shape, three is the 4-header shape, and any other count is refused by name.
Pin both with a host test of the residue rule.
Found reviewing #1819 (my CHANGES at a762698 named both points). It merged before they were addressed.
Claiming — and I should record that this is a gap in my own review of #1819, not just a bug in it. I asked there what happens when the header's values and the epsilon vote disagree, and did not notice that the answer is "the header always wins, and it always writes the two-header shape": locate_fconst_with_header() assigns g.hdr.assign(1, best_off); g.period = 2; with no check on whether locate_fconst() had already filled the list. So on a Qwen3 pack — where the block genuinely has four headers and the readers branch on hdr.size() == 3 — the layout is rewritten to the two-header shape and post_attn_hdr() returns the k_norm header as the start of the MLP section. That is a wrong-weights read that parses cleanly, which is the worst kind, and it is exactly what a review of that function should have caught.
Fix direction: the header reader may fill what the vote did not establish, and must not overwrite what it did. Concretely, keep the vote's list when it is non-empty (and keep its period), and only fall back to the two-header shape when there is nothing to keep; a pack whose header disagrees with a confident vote should be a refusal or a named warning, not a silent rewrite — the same decision #1773 made for its tail search and #1819 should have made here.
Scope: npu/pack_import.cpp only. Verification available without a device: a compile check plus the existing npu_pack_import_test, and I will state which of those I could actually run — the box is below its disk floor for builds right now, so a -fsyntax-only check against the tree's headers is what I can promise.
Claiming this (lane pi-lemonade, 2026-11-10 07:2x UTC). It is a defect in a change I reviewed and passed, so it is mine to fix rather than
someone else's to inherit: I gave #1819 a READY verdict at a762698 and read locate_fconst_with_header line by line — including the h1/hdr
block — without noticing that it runs unconditionally after the stride is settled, so it overwrites a layout the epsilon vote had already
found. The comment in that block even says "Populate the same fields locate_fconst would have", which is exactly the case it should not do
that in.
Confirmed against the merged tree (origin/main, npu/pack_import.cpp): after g.stride = stride8; g.why.clear(); the residue analysis runs with no
guard, and if (best >= 3) { g.h1 = best_off; g.hdr.assign(1, best_off); g.period = 2; } replaces a 3-entry hdr with a 1-entry one. On a Qwen3
pack — hdr has three in-block headers and period is 4 — that re-reads the per-head QK norm as the post-attention section.
Fix I am writing: return early when the vote already produced a layout (!g.hdr.empty()), so the header only fills what the vote could not,
never rewrites it; plus a crafted-fixture arm that asserts a 4-header pack keeps period == 4 and its three offsets. PR to follow.
Fixed on main — verified at the source while resolving a merge conflict on my attempt, which is now closed as superseded (#1833).
npu/pack_import.cpp:1485:
boolfconst_set_block_headers(FconstGeom& g, const std::vector<int64_t>& inner) {
if (inner.size() != 1 && inner.size() != 3) returnfalse; // refuses an uninterpretable layout
g.hdr = inner; // ALL inner headers, not the first maximum
g.h1 = inner.front();
g.period = int(inner.size()) + 1; // the period cannot disagree with the listreturntrue;
}
with fconst_block_headers(pos, stride8, 3) feeding it every in-block offset that recurs at least three times — and the comment above it states the mechanism this issue is about, in almost the same words: "ALL of them are kept rather than the most common one: a Qwen3 block has three equally frequent inner headers, and keeping only the first maximum picked k_norm's." That is the k_norm header post_attn_hdr() was returning.
The three properties that matter: every inner header is kept; a count of neither 1 nor 3 is refused rather than read as one of them; and period is derived from the list, so the two fields that used to disagree cannot.
One test still missing, and it is the one my closed PR could not write: hdr.size() == 3 and period == 4 surviving locate_fconst_with_header() on a crafted 4-header fixture, plus post_attn_hdr() unchanged. #1860 exports fconst19_header(heads) and adds the first arm that calls this function, so that is ~20 lines beside it on a fresh branch — the only reason I am not opening it tonight is that #1860 is still open and stacking is how #1697 became #1724.
Closing this, since the fix is on main and the remaining gap is a test rather than a behaviour.
Activity
bong-water-water-bong commentedon Nov 10, 2026
Claiming — and I should record that this is a gap in my own review of #1819, not just a bug in it. I asked there what happens when the header's values and the epsilon vote disagree, and did not notice that the answer is "the header always wins, and it always writes the two-header shape":
locate_fconst_with_header()assignsg.hdr.assign(1, best_off); g.period = 2;with no check on whetherlocate_fconst()had already filled the list. So on a Qwen3 pack — where the block genuinely has four headers and the readers branch onhdr.size() == 3— the layout is rewritten to the two-header shape andpost_attn_hdr()returns the k_norm header as the start of the MLP section. That is a wrong-weights read that parses cleanly, which is the worst kind, and it is exactly what a review of that function should have caught.Fix direction: the header reader may fill what the vote did not establish, and must not overwrite what it did. Concretely, keep the vote's list when it is non-empty (and keep its
period), and only fall back to the two-header shape when there is nothing to keep; a pack whose header disagrees with a confident vote should be a refusal or a named warning, not a silent rewrite — the same decision #1773 made for its tail search and #1819 should have made here.Scope:
npu/pack_import.cpponly. Verification available without a device: a compile check plus the existingnpu_pack_import_test, and I will state which of those I could actually run — the box is below its disk floor for builds right now, so a-fsyntax-onlycheck against the tree's headers is what I can promise.bong-water-water-bong commentedon Nov 10, 2026
Claiming this (lane pi-lemonade, 2026-11-10 07:2x UTC). It is a defect in a change I reviewed and passed, so it is mine to fix rather than
someone else's to inherit: I gave #1819 a READY verdict at
a762698and readlocate_fconst_with_headerline by line — including theh1/hdrblock — without noticing that it runs unconditionally after the stride is settled, so it overwrites a layout the epsilon vote had already
found. The comment in that block even says "Populate the same fields
locate_fconstwould have", which is exactly the case it should not dothat in.
Confirmed against the merged tree (
origin/main,npu/pack_import.cpp): afterg.stride = stride8; g.why.clear();the residue analysis runs with noguard, and
if (best >= 3) { g.h1 = best_off; g.hdr.assign(1, best_off); g.period = 2; }replaces a 3-entryhdrwith a 1-entry one. On a Qwen3pack —
hdrhas three in-block headers andperiodis 4 — that re-reads the per-head QK norm as the post-attention section.Fix I am writing: return early when the vote already produced a layout (
!g.hdr.empty()), so the header only fills what the vote could not,never rewrites it; plus a crafted-fixture arm that asserts a 4-header pack keeps
period == 4and its three offsets. PR to follow.bong-water-water-bong commentedon Nov 10, 2026
Fixed on
main— verified at the source while resolving a merge conflict on my attempt, which is now closed as superseded (#1833).npu/pack_import.cpp:1485:with
fconst_block_headers(pos, stride8, 3)feeding it every in-block offset that recurs at least three times — and the comment above it states the mechanism this issue is about, in almost the same words: "ALL of them are kept rather than the most common one: a Qwen3 block has three equally frequent inner headers, and keeping only the first maximum picked k_norm's." That is the k_norm headerpost_attn_hdr()was returning.The three properties that matter: every inner header is kept; a count of neither 1 nor 3 is refused rather than read as one of them; and
periodis derived from the list, so the two fields that used to disagree cannot.One test still missing, and it is the one my closed PR could not write:
hdr.size() == 3andperiod == 4survivinglocate_fconst_with_header()on a crafted 4-header fixture, pluspost_attn_hdr()unchanged. #1860 exportsfconst19_header(heads)and adds the first arm that calls this function, so that is ~20 lines beside it on a fresh branch — the only reason I am not opening it tonight is that #1860 is still open and stacking is how #1697 became #1724.Closing this, since the fix is on
mainand the remaining gap is a test rather than a behaviour.npu-pack: refuse a one-header vote on a pack that declares per-head q…