From b59516e8737ed0025f7b65bcc0513ec118d143c2 Mon Sep 17 00:00:00 2001 From: t Date: Mon, 9 Nov 2026 20:11:01 -0400 Subject: [PATCH 1/2] npu-pack: measure the 1.9 .fconst rule for hidden 3584 (two independent models agree), so the Qwen2.5-7B family imports (refs #1740) `kFconst19Rules` had no row for hidden 3,584 / 28 heads / 28 layers, so every Qwen2.5-7B pack stopped at "its block shape has not been measured". Measured with `tools/fconst19_measure.py` against the 1.8 twin of **two different models**, and both give the same row: Qwen2.5-7B-Instruct 2,219 of 3,830,638,912 bytes differ (5.79e-05 %) -> {3584, 28, 28, 7232, 8, 1, 1009664} Qwen2.5-Coder-7B-Instruct 28,271 of 3,830,638,912 bytes differ (7.38e-04 %) -> same row Two independent models of one geometry agreeing on `rot` is corroboration a single pair cannot give. `--probe-wrong` on the Coder pair separates the rotation by four orders of magnitude: rot+1 differs in 208,982,210 bytes (5.46 %), n+1 in 52,522,601 (1.37 %), against this row's 28,271. **The prefix is the one field this pair does not separate**, and the comment says so: `prefix+64` differs in 33,291 bytes, which is *under* the tool's 38,306-byte tolerance, so `prefix = 7,232` rests on the two models agreeing and on the plain Qwen2.5 arithmetic (`64 + 2*hidden`, no QK-norm term) rather than on the pair alone. A row comment that implied all seven fields were equally pinned would be the overstatement this table exists to avoid. The twins came from the `_npu_16K` naming, which is public while `_rai_1.8.0_npu` is 401 -- and the `_4K`/`_16K` Token `.fconst` of one model are byte-identical, so the variant suffix does not affect this measurement. Verified: `ninja npu_pack_import_test 1bit` rc 0; unit-only run `all npu-pack checks passed`; both pairs measured to the numbers above (the Coder pair with `--probe-wrong`); `tools/copyright.py --check` rc 0 with the tree staged; `git add ` only. Host-side, no device. --- npu/pack_import.cpp | 12 ++++++++++++ 1 file changed, 12 insertions(+) diff --git a/npu/pack_import.cpp b/npu/pack_import.cpp index 09479a3b5..8b6007bd5 100644 --- a/npu/pack_import.cpp +++ b/npu/pack_import.cpp @@ -403,6 +403,18 @@ constexpr Fconst19Rule kFconst19Rules[] = { // change the transform above notes -- so n=2/rot=1 is the minimal spelling of it (n=4/rot=2 and n=8/rot=4 are the // identical permutation, which is why the measurement tool prints the same residual for each). {3072, 32, 32, 10560, 2, 1, 5222400}, + // hidden 3,584 / 28 heads / 28 layers -- the Qwen2.5-7B family. Measured TWICE, against two different models' 1.8 twins, and + // both pairs give this row: Qwen2.5-7B-Instruct (2,219 of 3,830,638,912 bytes differ, 5.79e-05 %) and Qwen2.5-Coder-7B-Instruct + // (28,271 differ, 7.38e-04 %), each against its own `_rai_1.8.0_npu_16K` build. Two independent models of one geometry agreeing + // on `rot` is the corroboration a single pair cannot give, and `--probe-wrong` separates the rotation by four orders of + // magnitude on the Coder pair: rot+1 differs in 208,982,210 bytes (5.46 %), n+1 in 52,522,601 (1.37 %), against this row's + // 28,271. + // The prefix is the ONE field this pair does not separate: `prefix+64` differs in 33,291 bytes, which is under the tool's + // 38,306-byte tolerance -- so `prefix = 7,232` here rests on the two models agreeing and on the plain Qwen2.5 arithmetic + // (64 + 2*hidden, with no QK-norm term, unlike the Qwen3 line's +4,352), not on the pair alone. Said plainly because a row + // whose comment implies all seven fields were equally pinned would be the same kind of overstatement this table exists to + // avoid. The bundle is 8 tiles of 1,009,664 (8,077,312 B = 1,813,568 - 7,232, i.e. the post-attention header follows it). + {3584, 28, 28, 7232, 8, 1, 1009664}, // Mistral-7B-Instruct-v0.2 at hidden 4,096 / 32 heads / 32 layers: measured with tools/fconst19_measure.py against // amd/Mistral-7B-Instruct-v0.2_rai_1.8.0_npu_4K, 1,954 of 3,843,419,200 bytes differ (5.08e-05) -- content, not // layout. The prefix follows the Qwen3 line (64 + 2*hidden + 4,352 = 12,608), the bundle is 5 tiles of 2,297,856 From 1ec7bad4600a90edddb1b1ae6426b86b8926fcde Mon Sep 17 00:00:00 2001 From: t Date: Mon, 9 Nov 2026 20:44:03 -0400 Subject: [PATCH 2/2] npu-pack: the h3584 row's comment states arithmetic that adds up (review of #1758) The row's last sentence claimed `8 x 1,009,664 (8,077,312 B = 1,813,568 - 7,232)`, and 1,813,568 - 7,232 is 1,806,336 -- a number three orders from the one it was offered as evidence for. The 1,813,568 was the **h1536** pair's `header_off`, carried over from the measurement two comments earlier. Re-measured for this pair with `--json`: `header_off 9,094,208`, and the block works out as prefix 7,232 + 9 x tile 1,009,664 = 9,094,208 exact so the payload after the prefix is NINE tile slots -- eight that the rule rotates, and a ninth that is the gap the post-attention header sits behind. The comment now says that, and says that a reviewer caught the earlier version, so the next reader checks it rather than trusts it. Verified: `ninja npu_pack_import_test` rc 0; unit-only run `all npu-pack checks passed`; `tools/copyright.py --check` rc 0 with the tree staged. --- npu/pack_import.cpp | 7 ++++++- 1 file changed, 6 insertions(+), 1 deletion(-) diff --git a/npu/pack_import.cpp b/npu/pack_import.cpp index d10fb32db..0e83e1ba2 100644 --- a/npu/pack_import.cpp +++ b/npu/pack_import.cpp @@ -422,7 +422,12 @@ constexpr Fconst19Rule kFconst19Rules[] = { // 38,306-byte tolerance -- so `prefix = 7,232` here rests on the two models agreeing and on the plain Qwen2.5 arithmetic // (64 + 2*hidden, with no QK-norm term, unlike the Qwen3 line's +4,352), not on the pair alone. Said plainly because a row // whose comment implies all seven fields were equally pinned would be the same kind of overstatement this table exists to - // avoid. The bundle is 8 tiles of 1,009,664 (8,077,312 B = 1,813,568 - 7,232, i.e. the post-attention header follows it). + // avoid. + // The block's arithmetic, from the measurement rather than from a formula: prefix 7,232 + 9 x tile 1,009,664 = 9,094,208, + // which is exactly the `header_off` the tool reports for this pair -- so the payload after the prefix is NINE tile slots, + // of which the rule rotates eight and the ninth is the gap the post-attention header sits behind. (An earlier version of + // this comment subtracted the prefix from a per-block figure and produced a number three orders of magnitude off; the + // review of #1758 caught it. Written out here so the next reader can check it with a calculator.) {3584, 28, 28, 7232, 8, 1, 1009664}, // Mistral-7B-Instruct-v0.2 at hidden 4,096 / 32 heads / 32 layers: measured with tools/fconst19_measure.py against // amd/Mistral-7B-Instruct-v0.2_rai_1.8.0_npu_4K, 1,954 of 3,843,419,200 bytes differ (5.08e-05) -- content, not