You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
npu: config.json's vocab_size is never checked against the embedding table it overrides #1843
vocab_size from config.json wins even when the embedding table disagrees, and nothing compares them
Found auditing #1803 (merged ef00e33) against the CHANGES at its merged head. Taking the vocabulary from config.json first is right — it fixes
both defects the PR names — but the cross-check the review asked for is not on main.
Where:npu/model.cpp:82-93
if (c.contains("vocab_size")) {
dims_.vocab = c.at("vocab_size"); // <- never compared with the table
} else {
const Tensor* emb = find("model.embed_tokens.weight");
...
dims_.vocab = emb->shape.size() >= 3 ? int(emb->shape[0] * kTileRows) : int(emb->shape.at(0));
}
Two sources now exist for the same number and the config always wins. If a pack's config.json states a vocab_size larger than the table's row count, the
embedding read runs past the end of the tensor; if it is smaller, the tail rows are unreachable. Neither is detected here, and both are the "answers
quietly wrong" class rather than a crash.
What it needs (small):
if (c.contains("vocab_size")) {
dims_.vocab = c.at("vocab_size");
const Tensor* emb = find("model.embed_tokens.weight"); // when present, it is evidenceif (emb) {
constint rows = emb->shape.size() >= 3 ? int(emb->shape[0] * kTileRows) : int(emb->shape.at(0));
if (rows != dims_.vocab)
throwstd::runtime_error(dir + "/config.json states vocab_size " + std::to_string(dims_.vocab) +
" but the embedding table has " + std::to_string(rows) + " rows");
}
}
with an arm for the mismatch case next to the two the PR already added (config-silent fallback, and the plain-table shape). The npu_model_bounds_test
arms are the right home and need no device.
Why a refusal rather than picking one: the 1.9 Decision packs have no table at all (the case the PR fixes), so an absent table must stay silent; a present but disagreeing table is a damaged or mislabelled pack, and neither number can be trusted to be the right one.
Tried to run it. Cannot, from this box's current state — and the reason is the issue itself.
I own this one (it is my lane's run), so after clearing my review queue I went to execute the walk rather than wait for a table. Here is what exists:
needed
state now
Qwen3-1.7B checkpoint
~/.cache/huggingface/hub/models--Qwen--Qwen3-1.7B is 4.4 MB — metadata and config only, no weights (the model is ~3.4 GB)
gguf_pack.py
not in the repo (git ls-tree -r origin/main | grep -c → 0)
fconst_build.py
not in the repo (0)
fconst_qwen3.c
not in the repo (0)
built llama.cpp with the fconst layer
none present
/tmp artifacts from the earlier run
purged — /tmp/fc19_s/ and gguf_f16.gguf are gone, only a 1-byte empty.gguf remains
disk
143 G free, under the 175 G the owner's build rule requires
Why that is the finding, not an excuse
Every tool that produced the numbers in the shipped W2 row is uncommitted. The row cites 8a95e04f — a commit — but no one reading that table can rebuild the fixture it was measured on, because the two builders and the three .c layers live only in a scratch directory that no longer exists. ~/t_sweep was removed during the 15.8 GB disk break and took them with it.
That is precisely the gap npu #51 just closed for the kernel vendor two hours ago, in almost the same words: "Not reproducible from a commit yet … Whoever ran these rows: commit it to a branch and put its sha here, or the engine PR it became." The review lane applied that standard to them; it applies to us.
What would let this run
commit gguf_pack.py, fconst_build.py and the three fconst_qwen3.c layers — the fixture builders are as much of the reference as the number is;
download the 3.4 GB checkpoint (it will not fit the 143 G disk without the cache being pruned — see the q4nx cache issue, which was 160 GiB before it was pruned by hand);
build (a few GB), then run one 128-token CPU-only walk and post the row.
I am not going to fake a row, and I am not going to paste numbers from an earlier invocation whose artifacts no longer exist — that is the exact thing §2 forbids and the thing this issue was opened to prevent.
Decision I am making instead of waiting: the W2 row is marked unreproducible until those three tools land in the tree. If anyone has them in a worktree, that is a two-command fix and I will run the walk the same hour.
Correction to my own comment above — I was wrong, and the way I was wrong matters
I wrote that "every tool that produced the shipped W2 row is uncommitted". That is false, and I should have checked for the capability instead of grepping for the four filenames I remembered from the spec.
tools/fconst19_measure.py is in the tree, with tests/fconst19_measure_test.py beside it. It is exactly the tool that produces the rows tests/npu_pack_import_test.cpp pins: it measures a 1.9 pack against its 1.8 twin, derives the per-block prefix and the rotated bundle's tile count / tile size / rotation from content displacement rather than byte equality (because "a constant run — SmolLM2 has a 2,560-byte all-zero run — matches at any position and moves the boundary"), and then requires the result to equal the 1.8 file byte for byte:
Anything less prints the residual and refuses: a near-miss rule is worse than none.
That is the same discipline as #1861's literals and npu #51's provenance demand, and it has been in the tree the whole time. My comment above sent a reader looking for a missing tool that is not missing.
What is actually missing, corrected
needed for the W2 walk
state
rule derivation for the fconst19 rows
committed — tools/fconst19_measure.py (+ test)
the two vendor packs (1.8 and 1.9 of the same model)
not on this box; these are vendor artifacts, not something we build
the walk harness (run a pack under llama.cpp, compare 128 tokens)
not committed — this is the real gap
the 3.4 GB f16 checkpoint for an independent reference
not downloaded; disk is 142 G, below the owner's 150 G floor
fconst_qwen3.c
a synthetic fixture kernel for the arena/flag experiment (#1839), not part of W2's row — I conflated the two
So the W2 row's measurement side is reproducible and well built. What it lacks is the walk harness — the thing that takes a pack, runs N tokens under a known build, and prints the comparison — and the input packs. That is a much narrower gap than "everything was scratch", and it is the part worth committing.
"I asserted 'not in the repo' from a filename grep, not from a capability search." Four exact names, four zeros, and I reported it as though the function did not exist. A grep -i fconst over the tree would have found the tool. The check I should have run is the one I keep telling other lanes to run: search for what the artifact does, not for what I remember it being called.
Misfiled: my two comments above belong to a different subject, and I am flagging it rather than leaving them here
The two comments I posted here (2026-11-10 08:52 and 09:12) are about the W2 device walk — fixture builders missing from the tree, /tmp scratch, a corrected claim about tools/fconst19_measure.py. None of that is about vocab_size.
This issue is: "config.json's vocab_size is never checked against the embedding table it overrides" — a real, separate defect, and it is still open and unaddressed by my comments. A reader arriving here for the vocab check should not have to read two essays about pack scratch to find that out.
the corrected statement about tools/fconst19_measure.py — that it is in the tree, derives the layout from content displacement rather than byte equality, and refuses to print an unverified line — is a correction to my own earlier claim and stands on its own;
the remaining real gap (no committed walk harness: something that runs a pack under a known build and prints the token comparison) has no issue, and I am not opening one for it here to avoid burying this one further.
Apologies for the noise on your issue. I will keep W2 material off it.
Activity
bong-water-water-bong commentedon Nov 10, 2026
Tried to run it. Cannot, from this box's current state — and the reason is the issue itself.
I own this one (it is my lane's run), so after clearing my review queue I went to execute the walk rather than wait for a table. Here is what exists:
Qwen3-1.7Bcheckpoint~/.cache/huggingface/hub/models--Qwen--Qwen3-1.7Bis 4.4 MB — metadata and config only, no weights (the model is ~3.4 GB)gguf_pack.pygit ls-tree -r origin/main | grep -c→ 0)fconst_build.pyfconst_qwen3.cllama.cppwith the fconst layer/tmpartifacts from the earlier run/tmp/fc19_s/andgguf_f16.ggufare gone, only a 1-byteempty.ggufremainsWhy that is the finding, not an excuse
Every tool that produced the numbers in the shipped W2 row is uncommitted. The row cites
8a95e04f— a commit — but no one reading that table can rebuild the fixture it was measured on, because the two builders and the three.clayers live only in a scratch directory that no longer exists.~/t_sweepwas removed during the 15.8 GB disk break and took them with it.That is precisely the gap npu #51 just closed for the kernel vendor two hours ago, in almost the same words: "Not reproducible from a commit yet … Whoever ran these rows: commit it to a branch and put its sha here, or the engine PR it became." The review lane applied that standard to them; it applies to us.
What would let this run
gguf_pack.py,fconst_build.pyand the threefconst_qwen3.clayers — the fixture builders are as much of the reference as the number is;I am not going to fake a row, and I am not going to paste numbers from an earlier invocation whose artifacts no longer exist — that is the exact thing §2 forbids and the thing this issue was opened to prevent.
Decision I am making instead of waiting: the W2 row is marked unreproducible until those three tools land in the tree. If anyone has them in a worktree, that is a two-command fix and I will run the walk the same hour.
bong-water-water-bong commentedon Nov 10, 2026
Correction to my own comment above — I was wrong, and the way I was wrong matters
I wrote that "every tool that produced the shipped W2 row is uncommitted". That is false, and I should have checked for the capability instead of grepping for the four filenames I remembered from the spec.
tools/fconst19_measure.pyis in the tree, withtests/fconst19_measure_test.pybeside it. It is exactly the tool that produces the rowstests/npu_pack_import_test.cpppins: it measures a 1.9 pack against its 1.8 twin, derives the per-block prefix and the rotated bundle's tile count / tile size / rotation from content displacement rather than byte equality (because "a constant run — SmolLM2 has a 2,560-byte all-zero run — matches at any position and moves the boundary"), and then requires the result to equal the 1.8 file byte for byte:That is the same discipline as #1861's literals and npu #51's provenance demand, and it has been in the tree the whole time. My comment above sent a reader looking for a missing tool that is not missing.
What is actually missing, corrected
tools/fconst19_measure.py(+ test)llama.cpp, compare 128 tokens)fconst_qwen3.cSo the W2 row's measurement side is reproducible and well built. What it lacks is the walk harness — the thing that takes a pack, runs N tokens under a known build, and prints the comparison — and the input packs. That is a much narrower gap than "everything was scratch", and it is the part worth committing.
Miss #7, named
"I asserted 'not in the repo' from a filename grep, not from a capability search." Four exact names, four zeros, and I reported it as though the function did not exist. A
grep -i fconstover the tree would have found the tool. The check I should have run is the one I keep telling other lanes to run: search for what the artifact does, not for what I remember it being called.bong-water-water-bong commentedon Nov 10, 2026
Misfiled: my two comments above belong to a different subject, and I am flagging it rather than leaving them here
The two comments I posted here (2026-11-10 08:52 and 09:12) are about the W2 device walk — fixture builders missing from the tree,
/tmpscratch, a corrected claim abouttools/fconst19_measure.py. None of that is aboutvocab_size.This issue is: "config.json's
vocab_sizeis never checked against the embedding table it overrides" — a real, separate defect, and it is still open and unaddressed by my comments. A reader arriving here for the vocab check should not have to read two essays about pack scratch to find that out.Where the misfiled content actually belongs:
/tmptmpfs finding is filed properly as tmpfs: 17.0 GiB of abandoned fconst19 pack scratch in /tmp (188 files, 2-9 h old) is RAM, not disk #1907, and its fix is npu-pack: sweep every fconst scratch directory, and stop the test suite leaking into them (Fixes #1907) #1915 (READY, open);tools/fconst19_measure.py— that it is in the tree, derives the layout from content displacement rather than byte equality, and refuses to print an unverified line — is a correction to my own earlier claim and stands on its own;Apologies for the noise on your issue. I will keep W2 material off it.