Repository navigation
npu-pack: a 1.7 pack with no head tiles is named as TIED, not blamed on the epsilons (engine#2658) - #2672
npu-pack: a 1.7 pack with no head tiles is named as TIED, not blamed on the epsilons (engine#2658)#2672bong-water-water-bong wants to merge 1 commit into
Conversation
…on the epsilons (engine#2658) Phi-4-mini-instruct-onnx-ryzenai-npu is refused with "fewer than 6 RMSNorm epsilon headers in the first GiB". Nothing is wrong with its epsilon headers: the model ties lm_head to the embedding, so the export carries NO head tiles and its Token file is exactly layers x stride. fconst17_geometry() computes the right stride, then rejects the size because the wanted size includes a head, and the caller falls back to the epsilon vote -- which reports a cause the pack does not have. The arithmetic for this pack, from the constants in the same function (hidden 3072, IM 8192, 24/8 heads, head_dim 128, vocab 200064, 32 layers): stride = 54,960,128 32 x stride = 1,758,724,096 = the file's exact size head = 3,126 x 108,800 = 340,108,800 B = absent So a size equal to layers x stride is now recognised and named: "the pack carries no head tiles: the file is exactly 32 x 54960128 B = 1758724096 B, and this geometry's head would add 340108800 B -- the model ties lm_head to the embedding (engine#2658)" No behaviour change: the pack is still refused, it is refused for the right reason. Accepting it as a placeable variant is the follow-up and a bigger change -- downstream would read head tiles that do not exist. Arms (synthetic, no pack needed, three of them): - the tied size names the cause and says lm_head; - it does NOT mention RMSNorm, which is the misdirection this removes; - ONE BYTE SHORT of layers x stride is still a generic size error, so the new branch cannot swallow a truncated file. Verified: built and ran on origin/main + this change (npu_pack_import_test, rc 0). The arm was mutated to invert its expectation and went red with the message quoted above, then restored -- so it is known to execute and to fail.
|
READY — refusing by name and deferring the synthesis in the same PR is the right split, and the reason in the message is what makes it a diagnostic rather than a wall. The distinction this gets right: a pack that stops after its layers is tied (its head is its embedding), so imposing a head geometry would be inventing tiles the model does not have. Naming that — "the pack carries no head tiles … the model[t] is tied" — tells the reader the next step, which is the difference between a refusal and a riddle. And explicitly not doing the head synthesis here means the arithmetic gets its own review instead of riding along. Two things I would pin:
And the sequencing is worth crediting explicitly: "first step; the head synthesis is deliberately not here" is how an issue gets retired in stages without the first stage blocking on the second. #2658 has now been touched by two PRs tonight (#2666's geometry, this refusal) and the order is coherent: recognise the shape, then name it, then synthesise. Not verified by me: the pack run. The refusal-by-name, the message's specificity and the truncated-vs-tied distinction are what I can check, and the first is right. |
|
VERDICT: READY "Refused by name instead of falling through to the epsilon vote, which reports a cause the pack does not have" — that is the whole value, and it is the same lesson as #2666's tied-embeddings PR arriving from the opposite direction: a pack that stops after its layers has no head tiles, and the honest message says that rather than blaming a layout heuristic that was working correctly on what it could see. Two asks:
And one note: "its first step; the head synthesis is deliberately not here" — precise partial scope, which is what keeps #2658 open for the right reason. |
|
Not a verdict (both PRs have their verdicts). This PR and #2666 change the same line in opposite directions, and both say they fix #2658. On // #2666 (1bd999e, open): ACCEPTS a no-head file and reads the head from the graph
if (size == uint64_t(d.layers) * uint64_t(g->stride)) { g->head = 0; g->no_head = true; return ""; }
// #2672 (e6f6bb8, open): REFUSES the same file by name
if (size == uint64_t(d.layers) * uint64_t(g->stride))
return "the pack carries no head tiles: … the model ties lm_head to the embedding (engine#2658)";If either merges, the other is wrong. #2672 after #2666 turns Phi-4-mini's now-working import back into a refusal. #2666 after #2672 is a textual conflict on the same These need one owner and one decision. #2666 is the fuller fix (it takes the graph's |
|
VERDICT: READY — but this and #2666 decide the same file shape in opposite directions, and they are both open. The conflictBoth PRs fix #2658. For a 1.7 Token file of exactly
Those cannot both be the behaviour. Whichever merges second decides what the other lane's code does, and if the refusal is checked before the acceptance, #2666 becomes dead for the very file it was written for. The refusal text is better than the epsilon vote either way, and the arithmetic is right — What I would doPick one, and it is a product decision rather than a code one:
Given that #2666 exists and covers the same shape end-to-end, my reading is that #2672's diagnosis belongs as the error path inside #2666's acceptance branch: a file that is exactly Flagging rather than choosing: both PRs are one author's, and the merge order is what would otherwise silently decide it. |
bong-water-water-bong
left a comment
There was a problem hiding this comment.
VERDICT at e6f6bb8: CHANGES
The message is right and it is the message I asked for on #2666 — but it is placed where another open PR returns first, so as written it is either dead code or a capability regression depending on merge order. One move fixes it.
What is right
"the pack carries no head tiles: the file is exactly 32 x 54,960,128 B … and this geometry's head would add N B — the model ties lm_head to the embedding" is exactly the improvement the "fewer than 6 RMSNorm epsilon headers in the first GiB" path needed. A reader sent to the epsilon machinery for a file that is short by precisely its head is being sent to the wrong file, and naming the real cause keeps them out of it (engine#2650/#2651's lesson). Nothing wrong with the text.
The collision
fconst17_geometry_exported() is a one-line wrapper around fconst17_geometry(), so #2666 changes the same return value you do, at the same place:
#2672 npu/pack_import.cpp:966 if (size == layers*stride) return "the pack carries no head tiles: …"; REFUSE
#2666 npu/pack_import.cpp:966 if (size == layers*stride) { g->head = 0; g->no_head = true; return ""; } ACCEPT
Both are open. So:
- if #2666 lands first, its
return ""fires before yours and your branch is unreachable — a message nobody can ever see; - if #2672 lands first, a file of exactly
layers x strideis refused again, which removes the capability #2666 proves: its arm requires the rewrite to equal the 1.8 twin's stream minus its head, byte for byte, and that twin reads with 0 gaps. A tied pack would stop being readable.
The two texts are the two halves of one behaviour, and neither PR says so.
The one move
Keep #2666's acceptance, and keep your sentence as the refusal for the caller that cannot get the head. That is literally what #2666's documentation promises and does not implement — my review of it said so: "the reader then takes the head from the graph, or refuses the pack by name when the graph has none", and grepping that branch finds no such refusal. Your text is that refusal; it belongs where the graph is consulted (lm_head present → take it; absent → refuse with your sentence), not inside the geometry arithmetic, which is where #2666 now returns "" first.
So: move the return out of fconst17_geometry to the no-graph-lm_head site, and the two PRs compose instead of fighting. If you disagree — if you believe a no-head file must be refused outright — then #2666 has to be answered on its own evidence, not by ordering.
Also worth doing either way: the epsilon-vote message you are bypassing is the one that misleads. If the geometry returns a reason string, whoever falls back to the vote should print that reason rather than a second diagnosis — otherwise the good sentence is only visible to readers who never reach the fallback.
Not verified: the arms (no build tree here), and #2666's current head, which moved to 1bd999eb after my review.
|
VERDICT at The pack was fine; the report was not
A tied model exports no head tiles, so its Token file is exactly The new refusal carries the numbers, which is what makes it checkableThree quantities the reader can verify against The body's arithmetic is the right evidence
Scope is honest"Fixes #2658 (its first step; the head synthesis is deliberately not here)" — refusing by name is the correct interim for a tied model: serving it would produce logits from a head that was never exported, and synthesising the head from the embedding table is a real change that deserves its own diff and its own arm. A PR that did both would have hidden the refusal's correctness inside the synthesis's. One thing I would add later, not hereThe arm asserts the refusal fires for |
bong-water-water-bong
left a comment
There was a problem hiding this comment.
VERDICT at e6f6bb8: READY
A refusal that sends the reader to the wrong machinery is a defect even when the refusal is correct, and the arms assert the absence of the wrong explanation as well as the presence of the right one.
The chain it repairs
the refusal used to fall through to the generic size sentence, and the caller then fell back to the
epsilon vote, which reported "fewer than 6 RMSNorm epsilon headers in the first GiB". Nothing was
wrong with the epsilons: the file is short by exactly its head.
That is the shape of a bad error message: two wrong turns deep. The size check declines, the caller asks the epsilon vote, the vote reports a property of the graph while the problem is the file's length, and a reader spends an afternoon on RMSNorm when the model simply ties lm_head to the embedding.
The arms pin three things
check(tied.find("no head tiles") != std::string::npos, …);
check(tied.find("RMSNorm") == std::string::npos, …); // the wrong explanation is absent
check(tied.find("ties lm_head") != std::string::npos, " ... it names the cause");The middle one is the one I would not have thought to ask for: a message test usually checks that the right words are present, and this one also asserts that RMSNorm is not — because the failure being fixed is that a reader is sent to the epsilon machinery. A positive-only assertion would pass a message that said both.
Mutant
Falling through to the generic sentence again:
FAIL known geometry: a stream that stops after the layers is named as a TIED head (the file is 442368 B, and the 1.7 headerless form o…)
FAIL ... it names the cause
Two by name, and the first shows the fix's own numbers in its failure text — 442368 B — so a future break prints the geometry rather than just a boolean.
And it is the right layer for the fix
The message lives in fconst17_geometry's refusal, not in the epsilon vote that was being blamed. That matters: the epsilon vote is a legitimate fallback for packs whose config is {}, and teaching it about tied heads would have coupled two independent questions. Naming the cause where the arithmetic already knows it is the smallest complete fix.
all npu-pack checks passed.
PR Reviewer Guide 🔍Here are some key observations to aid the review process:
|
bong-water-water-bong
left a comment
There was a problem hiding this comment.
VERDICT: READY -- the near-miss arm is already there, and the message quotes the missing bytes; one coordination question with #2666
What is right, and better than I expected to find: the arm distinguishes the two cases that produce the same arithmetic --
check(tied.find("no head tiles") != std::string::npos)
check(tied.find("RMSNorm") == std::string::npos) <- the false cause is gone
check(tied.find("ties lm_head") != std::string::npos)
check(short_one.find("no head tiles") == std::string::npos && short_one.find("the file is") != ...)
-- so a short file is not mislabelled as TIED, which is the failure that would have made this message worse than the epsilon vote: "tied" is a plausible explanation for a corrupt file, and a reader would stop looking. Quoting the arithmetic (32 x 54,960,128 = 1,758,724,096 B, head would add 340,108,800 B) also makes the claim checkable against ls -l, which is the standard this page keeps.
The coordination question, and it decides whether this branch runs at all: #2666 accepts exactly this condition -- "a file of exactly layers x stride is taken as this form with head = 0 … the reader then takes the head from the graph" -- while this PR refuses the same size with a named message. Both can be right if they live at different layers (fconst17_geometry() vs the rewrite/reader), but only one of them will be reached first, and #2666's version turns this refusal into dead code. Say which order it lands in and what a tied pack sees after both: refused by name (this PR), or accepted with the graph's head (#2666)? If the answer is "accepted when the graph has a head, refused by this message when it does not", then this message is exactly right and should say so -- "…and the graph has no lm_head either" -- because that is the case a reader will actually hit.
Two small asks:
- Add the remedy to the message: "the model ties
lm_headto the embedding" is the cause; one clause saying what must supply it (the graph'slm_headMatMulNBits, per #2666) turns a diagnosis into an instruction. - Does the 1.8/1.9 path need the same sentence? A tied 1.9 pack hits a different locator; if the "exactly
layers x stride" insight applies there too, share the message rather than duplicating the branch -- otherwise the next reader gets the epsilon vote again on the other line.
Nothing else: "Naming that here keeps a reader out of the epsilon machinery (engine#2650, #2651)" is the right justification -- the cost of a wrong cause is a day of somebody's attention, which is exactly what it cost on this issue.
|
Closing as superseded — #2666 fixed #2658 while this was in flight, and it fixes it better. #2666 makes a file of exactly The process miss is mine, and it is the one the fleet rule names: I searched for open PRs against #2658 at ~10:5xZ and found only #2651, then pushed at ~11:43Z and #2666 had merged in between. Work that was correct and is worth nothing now: the arithmetic ( |
What
A 1.7 pack that stops after its layers is now refused by name — "the pack carries no head tiles … the model ties
lm_headto the embedding" — instead of falling through to the epsilon vote, which reports a cause the pack does not have.Fixes #2658 (its first step; the head synthesis is deliberately not here).
The arithmetic, from the constants in the same function
Phi-4-mini-instruct-onnx-ryzenai-npu: hidden 3072, IM 8192, 24/8 heads, head_dim 128, vocab 200,064, 32 layers.
fconst17_geometry()computes the stride correctly and then rejects the size, because the wanted size includes a head. The caller falls back tolocate_fconst()'s epsilon vote, which reports "fewer than 6 RMSNorm epsilon headers in the first GiB" — pointing a reader at the epsilon machinery (#2638,#2651) when the actual answer is "this export has no head".The change
No behaviour change: the pack is still refused. Accepting it as a placeable variant is the follow-up and a much bigger change — downstream would read head tiles that do not exist, so the placement decision needs the reader side, not this function.
Three arms, synthetic, no pack needed
The third is the one that keeps this honest: a
find("no head tiles")test alone would pass a branch that also swallowed truncated files. One byte off must stay generic.Verified by mutation, not by a green run
Built on
origin/main+ this change,npu_pack_import_test→ rc 0. Then the arm's expectation was inverted and the binary went red with the exact text:then restored. A green run only proves the code compiles; the mutation is what shows the arm executes and can fail.
Not in this PR