From 91815b27b6d1e36a6dd9778c1781a6a408c44f51 Mon Sep 17 00:00:00 2001 From: pi-inferzo Date: Mon, 9 Nov 2026 07:41:37 -0400 Subject: [PATCH] npu-pack: measure the 1.9 Token fconst rule for hidden 3072 / 32 heads / 32 layers (2 packs run OOB) The table refused this geometry by name: 'its block shape has not been measured for hidden 3072, 32 heads, 32 layers ... the 1.8 pack of the same model is what establishes it.' So it was measured that way, against Phi-3.5-mini's own 1.8 build (the `m1.0` twins are gated for this lane), and the row is: {3072, 32, 32, prefix 10560, n_tiles 2, rot 1, tile 5222400} Why this is a rule and not a fit -- the residual distribution, not its size: residual 26,145 of 2,036,067,648 (1.284e-5), tail 0 per block: 0 0 0 0 0 0 0 0 0 0 9403 0 0 0 0 0 0 0 9128 7499 0 0 0 0 0 0 0 0 115 0 0 0 28 of 32 blocks byte-identical; every differing byte is inside blocks 10, 18, 19, 28 That is content, not layout (the same signature the Qwen3-1.7B row records: 535 bytes, all in layer 24). The wrong end of the same pair separates by four orders of magnitude: any other rotation differs in 307,097,372 bytes (15 %) across all 32 blocks. `n=2/rot=1`, `n=4/rot=2` and `n=8/rot=4` are the identical permutation (`rot = n/2` is a half-swap), which is why tools/fconst19_measure.py prints one residual for all three; n=2 is the minimal spelling. End to end, on a DIFFERENT model at the same geometry, which is what makes it a rule and not a curve fit: amd/Phi-3-mini-128k-instruct_rai_1.9.0_npu import clean (stride 61923520, 64/64 RMSNorm anchors, 6144/6144 attention tiles + 5632/5632 MLP groups byte-exact) serve: 2/2 greedy runs byte-identical -> ' Neil Armstrong. He was an astronaut and the commander of the Apollo 11 mission.' amd/Phi-3.5-mini-instruct_rai_1.9.0_npu same, 2/2 identical -> ' an American astronaut named Neil Armstrong. He was part of the Apollo 11 mission, which landed' Both ran with kernels-phi35-b8, whose four design shapes (qkv 3072x9216, o 3072x3072, gu 3072x16384, d 8192x3072) are exactly this geometry's. Tool tolerance is left alone: this pair's residual (1.284e-5) sits just above the tool's 1e-5 gate, 4.8x above the worst legitimate residual it had seen and 115x below the mildest wrong rule. That gate is the tool author's calibrated property, so the measurement is posted on #1503 for them to move rather than loosened here. Refs #1503. --- npu/pack_import.cpp | 10 ++++++++++ 1 file changed, 10 insertions(+) diff --git a/npu/pack_import.cpp b/npu/pack_import.cpp index eb8118c4d..6a5bffc63 100644 --- a/npu/pack_import.cpp +++ b/npu/pack_import.cpp @@ -299,6 +299,16 @@ constexpr Fconst19Rule kFconst19Rules[] = { // prefix as Qwen3-1.7B at this hidden size (8,512), a different bundle (5 tiles of 591,872, not 3 of 1,183,744), // and 1,857 of 686,518,336 bytes differ (2.7e-6). This row is the one that disproved the QK-norm explanation above. {2048, 32, 16, 8512, 5, 1, 591872}, + // Phi-3.5-mini-instruct at hidden 3,072 / 32 heads / 32 layers (and Phi-3-mini-128k, the same phi3 geometry; the + // table is keyed by shape, and the Llama row above is what established that the prefix is a property of the shape). + // Measured against Phi-3.5-mini's own 1.8 build (`_rai_1.8.0_npu_4K`) by whole-file normalisation: 28 of the 32 + // blocks come out byte-identical and the four that differ hold all 26,145 differing bytes of 2,036,067,648 + // (1.28e-5), i.e. content in four layers, not layout. The separation that makes this a rule rather than a fit is + // the wrong-rotation end: any rotation other than this one differs in 307,097,372 bytes (15 %) across all 32 + // blocks, a factor of 10,000. The two halves of the bundle are swapped relative to 1.8 -- the same q/k/v order + // change the transform above notes -- so n=2/rot=1 is the minimal spelling of it (n=4/rot=2 and n=8/rot=4 are the + // identical permutation, which is why the measurement tool prints the same residual for each). + {3072, 32, 32, 10560, 2, 1, 5222400}, }; // The header a 1.9 block carries: 00 00 7f ff once per head. Its length being the head count is what makes