From 2c2f28c270f8fe75a6a35cd5cb7df96a6145b032 Mon Sep 17 00:00:00 2001 From: t Date: Tue, 10 Nov 2026 01:56:55 -0400 Subject: [PATCH 1/2] npu-pack: take a 1.9 pack's stride and second-header offset from its own header, not from the epsilon-anchor vote (refs #1740) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit The first import of an h3584 pack (Qwen2-7B, 8.95 GB, geometry exactly the family's) refused: fconst: layer blocks not located: the epsilon headers do not repeat with a single period while `tools/fconst19_measure.py` reported the pack's structure exactly -- `header_off 9,094,208`, `stride9 126,098,736`, byte-identical to the two packs the h3584 rule was measured on. The pack was fine; the locator was not. **Why the vote loses.** `locate_fconst()` derives the stride by voting on `pos[i+P]-pos[i]` over epsilon anchors (bf16 1e-6 + ten zeros) and demanding a 66 % mode. In 1 GiB of these weights, eps-SHAPED byte pairs are common: this pack yields 21 positions for 28 layers and the right stride (`126,098,624` = `126,098,736 - 28*4`) wins only **52.6 %** of lag-2 pairs. Under the bar -> refusal. **Why re-deriving it at all was the mistake.** `canonical_token_fconst()` has already computed the same stride from the pack's 112-byte per-head header (`fconst19_detect`, then `stride8 = stride9 - hdr_len`) to build the very copy the locator then scans. A 112-byte exact structure repeated 28 times is a far stronger witness than a 12-byte coincidence, so the header now wins and the vote is the fallback -- which is what the 1.8 / R1-Distill line still uses, where it works. The second field has to come with it: `h1` (the second eps header inside a block) decides whether the payload reader sees 2 headers per block or 4, and leaving it 0 made every MLP projection refuse. With the stride known, the anchors' residues `pos mod stride` repeat across all 28 blocks, so their mode (excluding 0) is `h1` -- no dominance test needed. Measured on `amd/Qwen2-7B_rai_1.9.0_npu`, same command before and after: before: fconst: layer blocks not located … incomplete: 280 container 1.1 GB after: fconst: stride 126098624, second header 16161856, 56/56 RMSNorm anchors MLP: 9856/9856 .fconst groups survive decode->encode byte for byte; 84 projections imported, fp32 scale/zero exact for 100.00 % of groups incomplete: 196 (attention only) container 5.4 GB Verified: `npu_pack_import_test tests/golden` -> all checks pass; with the reference 1.9 pack -> `all npu-pack checks passed (including the pack leg)`; `tools/copyright.py --check` rc 0 with the tree staged; `git add ` only. Not fixed here, stated so the next step is not a surprise: the **attention** payload still refuses on this pack with "the layer block has 1 epsilon headers, and this importer reads blocks of 2 … or 4", while MLP imports byte-exact. That is a block-shape question (this pack's payload between `prefix` and `h1` is 16 tile-slots; the h3584 rule records an 8-tile bundle), not a locator question, and it needs the same pair-measurement treatment the rule row got -- which is why it is a separate change. --- npu/pack_import.cpp | 52 +++++++++++++++++++++++++++++++++++++++++++-- 1 file changed, 50 insertions(+), 2 deletions(-) diff --git a/npu/pack_import.cpp b/npu/pack_import.cpp index 19c171787..9654ff693 100644 --- a/npu/pack_import.cpp +++ b/npu/pack_import.cpp @@ -1309,6 +1309,54 @@ int intermediate_size_from_graph(const std::string& model_path, int64_t hidden) if (fused.size() != 1) return 0; return int(*fused.begin()); } +// engine#1740: a 1.9 pack's stride is KNOWN before the eps vote -- canonical_token_fconst() derives it from the 112-byte +// per-head header to build its copy -- but `locate_fconst`'s epsilon-anchor vote can still lose when the payload is dense in +// eps-SHAPED bytes (bf16 1e-6 followed by ten zeros is not rare in 4-bit weights). Measured on Qwen2-7B 1.9: the right stride +// won only 52.6 % of lag-2 pairs, under the 66 % bar, so a 3.8 GB pack structurally identical to two measured ones was refused +// with "the epsilon headers do not repeat with a single period" while its own header reported the stride exactly. +// So ask the header first on the SOURCE file (the canonical copy has the 1.9 header removed by construction), and keep the eps +// vote as the fallback for packs with no 1.9 header -- the 1.8 / R1-Distill line, where it works today. +FconstGeom locate_fconst_with_header(const std::string& tok, const std::string& pack_dir, const Dims& dims) { + FconstGeom g = locate_fconst(tok, dims.eps_bits); + const std::string src = find_token_fconst(pack_dir); + if (src.empty()) return g; + Fconst fc = open_fconst(src); + int64_t hdr_off = 0, stride9 = 0; + const bool is19 = fconst19_detect(fc, dims.heads, dims.layers, &hdr_off, &stride9); + unmap_file(fc.map, fc.size); + if (!is19) return g; // no 1.9 header: the eps vote is the only witness, keep it + const int64_t stride8 = stride9 - int64_t(dims.heads) * 4; + if (stride8 <= 0) return g; + if (g.stride != 0 && g.stride != stride8) { + // Two witnesses that disagree: say so rather than pick. They cannot both be right, and a wrong stride reads weights + // from the wrong place instead of failing cleanly. + g.why = "the 1.9 header gives stride " + std::to_string(stride8) + " and the epsilon anchors give " + + std::to_string(g.stride) + "; refusing rather than choosing"; + g.stride = 0; + return g; + } + g.stride = stride8; + g.why.clear(); + // The second header offset (`h1`) is what tells the payload reader whether a block holds 2 eps headers or 4, so the eps + // vote failing must not leave it at 0: with the stride known, the anchors' offsets *within* a block repeat, and the mode + // of `pos mod stride` (excluding 0, the block start) is the second header. This is the same pool of anchors the vote + // could not win on, used for a question that does not need a dominance test -- 28 blocks each contribute their residue. + const std::vector pos = eps_header_positions(tok, size_t(1) << 30, dims.eps_bits); + if (!pos.empty()) { + std::map off; + for (const int64_t q : pos) { + const int64_t r = q % stride8; + if (r > 0 && r < stride8) ++off[r]; + } + int best = 0; + int64_t best_off = 0; + for (const auto& [o, n] : off) + if (n > best) { best = n; best_off = o; } + if (best_off > 0) g.h1 = best_off; + } + return g; +} + std::string find_token_fconst_exported(const std::string& pack_dir) { return find_token_fconst(pack_dir); } // engine#1638: the geometry READ (not just its dims), so a test can assert where each number came from -- the @@ -2252,7 +2300,7 @@ Plan plan(const std::string& pack_dir) { "payload is not 64-row tiles of this shape"; p.mlp_note = p.attn_note; } else { - const FconstGeom g = locate_fconst(tok, p.dims.eps_bits); + const FconstGeom g = locate_fconst_with_header(tok, pack_dir, p.dims); if (g.stride == 0) { p.attn_note = "layer blocks not located: " + g.why; p.mlp_note = p.attn_note; @@ -3397,7 +3445,7 @@ Report import_pack(const std::string& pack_dir, const std::string& out_dir, cons { std::string path_why; const std::string path = materialise_token_fconst(pack_dir, p.dims); - const FconstGeom g = locate_fconst(path, p.dims.eps_bits); + const FconstGeom g = locate_fconst_with_header(path, pack_dir, p.dims); // refusals decided while reading the payload (a tile that does not survive its own round trip, // or an o_proj bias the container has nowhere to put); they land before anything is written std::vector late; From a762698cb6b64530823153f4dfdbd7dbec27c86e Mon Sep 17 00:00:00 2001 From: t Date: Tue, 10 Nov 2026 02:02:34 -0400 Subject: [PATCH 2/2] npu-pack: the header-derived stride must also populate the header LIST, or every attention projection refuses (refs #1740) Follow-up in the same change: with `stride` and `h1` set but `hdr`/`period` left empty, the payload reader classified the pack as "1 epsilon header" (`hdr.size() + 1`) and refused all 196 attention tensors while the MLP imported byte-exact. `hdr` is the in-block header list and its SIZE is how a pack is classified as a 2-header (q/k/v/o in one payload) or 4-header (Qwen3: q/k/v, two per-head QK norms, o_proj) block, so the header path has to fill it. Measured on `amd/Qwen2-7B_rai_1.9.0_npu`, same command: incomplete 280 -> 196 (stride + h1) -> 0 container 1.1 -> 5.4 -> 6.0 GB attention: 3584/3584 .fconst tiles survive decode->encode byte for byte; 112 projections + 84 bias tensors imported, fp32 scale/zero exact for 100.00 % of groups MLP: 9856/9856 groups byte-exact; 84 projections imported fconst: stride 126098624, second header 16161856, 56/56 RMSNorm anchors found incomplete: 0 tensors The threshold is "at least 3 blocks agree" rather than `layers/2`: the anchor scan window is the first GiB, about 8 blocks at this geometry, so the right residue repeats ~8 times against 1-2 for a payload coincidence -- requiring half the layers silently dropped the real one and reverted this to 280. Verified: `npu_pack_import_test tests/golden` clean; with the 0.5B 1.9 pack `all npu-pack checks passed (including the pack leg)`; `tools/copyright.py --check` rc 0 with the tree staged. --- npu/pack_import.cpp | 14 +++++++++++++- 1 file changed, 13 insertions(+), 1 deletion(-) diff --git a/npu/pack_import.cpp b/npu/pack_import.cpp index 9654ff693..06ea1f825 100644 --- a/npu/pack_import.cpp +++ b/npu/pack_import.cpp @@ -1352,7 +1352,19 @@ FconstGeom locate_fconst_with_header(const std::string& tok, const std::string& int64_t best_off = 0; for (const auto& [o, n] : off) if (n > best) { best = n; best_off = o; } - if (best_off > 0) g.h1 = best_off; + if (best_off > 0) { + // Populate the same fields `locate_fconst` would have: `hdr` is the in-block header list and the reader + // classifies a pack as a 2-header (one inner anchor) or 4-header (three) block from its SIZE. Setting only + // h1 left `hdr` empty, so a pack whose anchors could not win the vote was classified as "1 epsilon header" + // and every attention projection refused while the MLP imported byte-exact -- measured on Qwen2-7B 1.9. + // The scan window is the first GiB (about 8 blocks of this geometry), so the right residue repeats ~8 times + // against 1-2 for a payload coincidence. Requiring `layers/2` was too strict and silently dropped the real one. + if (best >= 3) { // the residue repeats across blocks: a real section boundary + g.h1 = best_off; + g.hdr.assign(1, best_off); // 2-header block: q/k/v/o in one payload (Qwen2/Llama) + g.period = 2; + } + } } return g; }