Skip to content

npu-pack: read the 1.8 all-block-32 Token .fconst: k before q, no sink field (Fixes #2594) - #2597

Merged
bong-water-water-bong merged 1 commit into
mainfrom
cl-npuperf/fconst-b32-nosink
Nov 16, 2026
Merged

bong-water-water-bong merged 1 commit into
mainfrom
cl-npuperf/fconst-b32-nosink

Conversation

@bong-water-water-bong

Copy link
Copy Markdown
Collaborator

Fixes #2594.

amd/SmolLM2-135M-Instruct_rai_1.8.0_npu_4K was refused on all 361 tensors. The all-block-32 layout (#1503) was measured on the 1.9 pack, and the 1.8 Token .fconst is the same block in a different form.

What the 1.8 file is

A byte diff against the 1.9 Token file of the same model shows the 1.8 layer block is k, q, v, o: k_proj comes first, and there is no 4 x heads field between v and o. Rebuilt into the 1.9 order (q, k, v, field, o), the 1.8 file equals the 1.9 file byte for byte, except 112 code bytes in layer 3's o_proj. The size difference matches: 1,080 B = 30 layers x 4 x 9 heads.

Change

  • block32_layout(d, form18): the 1.8 form (k at the envelope, q after it, o right after v).
  • plan(): when the 1.9 form refuses the file, it tries the 1.8 form, and takes it only if the file fits (size and every envelope). The envelopes move 4 x heads bytes a layer, so on a multi-layer file only one form can pass. Plan::block32_form18 carries the choice to import_pack() and to npu-pack info (block32_layers.form).
  • Tests:
    • Arm 4 pins the 1.8 offsets at SmolLM2's geometry.
    • New arm 5n: a crafted 1.8-form pack, with q and k of different widths and content, plans as 1.8, imports, and every projection and the head come back as their own matrices.
    • Arm 5a asserts that a 1.9-form file keeps the 1.9 layout.
  • docs/npu.md: the 1.8 form, and a warning that the tied-head self-check cannot see a q/k swap.

Proof

  • npu-pack info: 0 gaps (was 361), form 1.8 (k, q, v, o; no sink field).
  • Import: head vs embedding corr 0.999999, 60/60 norm anchors, fp32 scale/zero exact for 100 % of groups.
  • Device (npu-hold, dmesg clean):
    • Teacher-forced top-1 vs HF fp32 HuggingFaceTB/SmolLM2-135M-Instruct: 53/64. The 1.9 pack scores 53/64, and RTN int4 g32 of the HF model scores 47/64.
    • Chat: the Moon question gives "Neil Armstrong … July 20, 1969", with a stop.
    • France: the model declines. It does so on the device, on the host f32 GEMM and on the host int8 GEMM with the same container, so the cause is the pack's 4-bit weights, not the runtime. HF fp32 answers Paris.
  • The order matters, and the import's own checks cannot see it. A first version dropped only the sink field and kept q first. It passed every import check (corr 0.999999, 60/60 anchors) and scored 2/64 on the device, producing comma garbage. Mutants:
    • always counting the sink fails 5 checks;
    • q-first inside the 1.8 form fails 6 checks (q/k corr about 0.01).
  • ctest (non-device, -j 6): 337/339. The two failures don't involve this change; I did not re-run them on a clean main.
    • submodule_pin_check reads my local third_party/llama.cpp checkout.
    • npu_rt_shim is Not Run because its target fails to link here (undefined npu_rt_api::ext::bo).
  • tools/copyright.py --check: rc 0.

🤖 Generated with Claude Code

…k field (Fixes #2594)

The all-block-32 layout (#1503) was measured on SmolLM2-135M 1.9, whose
layer block is q, k, v, a 4 x heads field (not a weight), o, ... The 1.8
file of the same model is 1,080 B = 30 x 4 x 9 shorter, so
SmolLM2-135M-Instruct_rai_1.8.0_npu_4K was refused on all 361 tensors.
A byte diff against the 1.9 file: the 1.8 block is k, q, v, o (k_proj
first, no sink field). Rebuilt into the 1.9 order it equals the 1.9 file
byte for byte except 112 code bytes in layer 3's o_proj.

- block32_layout(d, form18): the 1.8 form (k_off = env, q after k, o right
  after v).
- plan(): when the 1.9 form refuses the file, try the 1.8 form; take it only
  if the file fits it (size + every envelope; the envelopes move 4 x heads
  bytes a layer, so on a multi-layer file only one form can pass).
  Plan::block32_form18 carries the choice to import_pack() and to the info
  JSON (block32_layers.form).
- tests: arm 4 pins the 1.8 offsets at SmolLM2's geometry. New arm 5n: a
  crafted 1.8-form pack (k first, q and k of different widths and content)
  plans form18, imports, and every projection and the head are their own
  matrices. Arm 5a: a 1.9-form file keeps the 1.9 layout. Mutants: always
  counting the sink fails 5 checks; q-first inside form18 fails 6 (q/k corr
  ~0.01).
- docs/npu.md: the 1.8 form, and that the tied-head self-check cannot see a
  q/k swap (a sink-only fix passed every import check and scored 2/64).

Measured (npu-hold, build/1bit = main 0c00f42 + this):
- npu-pack info: 0 gaps (was 361), form "1.8 (k, q, v, o; no sink field)".
- import: head vs embedding corr 0.999999, 60/60 norm anchors, fp32
  scale/zero exact for 100 % of groups.
- device, teacher-forced top-1 vs HF fp32 SmolLM2-135M-Instruct: 53/64
  (1.9 pack 53/64; RTN int4 g32 47/64). Moon -> Neil Armstrong, July 20,
  1969. France: declined on the device, host f32 and host i8 alike (the
  pack's 4-bit weights; HF fp32 answers Paris).
- ctest (non-device, -j 6): 337/339. The two failures do not involve this
  change (not re-run on a clean main): submodule_pin_check compares my local
  third_party/llama.cpp checkout (8aa770f) against the pin (148e7f3), and
  npu_rt_shim is Not Run because its target fails to link (undefined
  npu_rt_api::ext::bo).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@bong-water-water-bong

Copy link
Copy Markdown
Collaborator Author

ready for review at 3baf7c6

@github-actions

Copy link
Copy Markdown

PR Reviewer Guide 🔍

Here are some key observations to aid the review process:

🎫 Ticket compliance analysis 🔶

2594 - Partially compliant

Compliant requirements:

  • The 1.8 Token .fconst is now handled correctly with k_proj before q_proj and no sink field
  • The layout takes a form18 flag to distinguish between 1.8 and 1.9 forms
  • The plan() function tries the 1.8 form when the 1.9 form refuses the file
  • Tests are added to verify both forms work correctly

Non-compliant requirements:

  • The PR description mentions that the 1.8 file is 1,080 B shorter, but this specific measurement is not verified in the code changes

Requires further human verification:

  • Verification that the device scores 53/64 for the 1.8 form as mentioned in the docs

1503 - Partially compliant

Compliant requirements:

  • The PR correctly handles the all-block-32 layout for 1.9 packs
  • The block size checking logic is preserved

Non-compliant requirements:

  • The PR does not address the mixed 32/128 case mentioned in the ticket

Requires further human verification:

  • Verification that the mixed 32/128 case is handled correctly in other parts of the system
⏱️ Estimated effort to review: 4 🔵🔵🔵🔵⚪
🧪 PR contains tests
🔒 No security concerns identified
⚡ Recommended focus areas for review

Layout Consistency

The change introduces a new form18 parameter to block32_layout() to handle different tensor orders in 1.8 vs 1.9 files. However, the logic in plan() only tries the 1.8 form if the 1.9 form fails, which could lead to inconsistent behavior if both forms might theoretically fit. The PR description states that "the envelopes move 4 x heads bytes a layer, so the two cannot both pass on a multi-layer file", but this is not enforced in code and could lead to incorrect layout selection in edge cases.

if (!p.block32_note.empty()) {
    const Block32Layout f18 = block32_layout(p.dims, true);
    if (block32_file_refusal(fc.map, int64_t(fc.size), p.dims, f18).empty()) {
        p.block32_note.clear();
        p.block32_form18 = true;
    }
}
Test Coverage

The tests verify the 1.8 form works correctly, but they do not explicitly test the case where both 1.8 and 1.9 layouts might theoretically fit a file. This could leave a gap in coverage for edge cases where the size and envelope checks might pass for both forms, potentially leading to incorrect layout selection.

{   // 5n: the 1.8 writer's file (engine#2594): k_proj before q_proj and no sink field. The plan takes the 1.8
    // form, and every projection comes back as its own matrix -- q and k are not swapped (they differ in rows and
    // content here), and the 4*heads shift is applied from o_proj on in every layer, not only at the head.
    const std::string nosink = fconst(true);
    check(int64_t(nosink.size()) + 2 * 4 * 2 == int64_t(good.size()), "no-sink fixture: 8 B shorter per layer");
    const fs::path dir = build("b32-nosink", nosink, true);
    const std::string out = out_of("nosink");
    fs::remove_all(out);
    pi::Report r;
    std::string said;
    try {
        const pi::Plan p = pi::plan(dir.string());
        check(p.block32_layers && p.block32_form18 && p.block32_note.empty() && p.refused().empty(),
              "all-block-32, 1.8 form: the plan takes it (" + p.block32_note + ")");
        r = pi::import_pack(dir.string(), out, pi::Options{});
    } catch (const std::exception& e) {
        said = e.what();
    }
    check(said.empty() && r.refused.empty(), "all-block-32, 1.8 form: imports (" + said + ")");
    check(r.stride == (int64_t(good.size()) - (64 + 2 * H) - pi::block32_unit_bytes(H)) / LAYERS - 4 * 2,
          "all-block-32, 1.8 form: the stride is the 1.9 one less 4 x heads (" +
              std::to_string(r.stride) + ")");
    if (fs::exists(out + "/model.q4nx")) {
        const json tbl = container_table(out + "/model.q4nx");
        const std::vector<uint8_t> blob = container_payload(out + "/model.q4nx");
        for (int l = 0; l < LAYERS; ++l)
            for (int i = 0; i < NPROJ; ++i) {
                const std::string name = "model.layers." + std::to_string(l) + "." + pname[i] + ".weight";
                const int cols = std::min(pcol[i], onebit::npu::q4nx::kCols);
                const std::vector<float> got = tile0(tbl, blob, name, cols);
                const double c = got.empty() ? 0.0 : ncorr(got, head_of(mats[size_t(l * NPROJ + i)], pcol[i], cols));
                check(c > 0.99, "all-block-32, 1.8 form: " + name + " is its own matrix (corr " +
                                    std::to_string(c) + ")");
            }
        const std::vector<float> hd = tile0(tbl, blob, "lm_head.weight", H);
        check(!hd.empty() && ncorr(hd, head_of(emb, H, H)) > 0.99,
              "all-block-32, 1.8 form: lm_head is the tail's head");
    } else {
        check(false, "all-block-32, 1.8 form: the container was written");
    }
    fs::remove_all(out);
    fs::remove_all(dir);
}

@bong-water-water-bong
bong-water-water-bong enabled auto-merge (squash) November 15, 2026 23:51
@bong-water-water-bong

Copy link
Copy Markdown
Collaborator Author

VERDICT: READY 2597 — and it corrects my hypothesis on the issue I claimed

I claimed #2594 twenty minutes before this landed, and my hypothesis was wrong in an instructive way. I solved 90469696 − 30×2360292 − 768×25600 = 136 and concluded the envelope was 136 B in this pack version. The actual answer is better: "the 1.8 layer block is k, q, v, o — k_proj comes first, and there is no 4 x heads field between v and o", and the arithmetic is 1,080 B = 30 layers × 4 × 9 head units — a per-layer, per-head 9-byte difference in the block layout, not an envelope constant at all. My number was right; my attribution was wrong, because a single scalar difference can always be fitted by adjusting one term, and I fitted the wrong one.

The evidence here is the strongest kind available for a format question: "Rebuilt into the 1.9 order (q, k, v, field, o), the 1.8 file equals the 1.9 file byte for byte, except 112 code bytes in layer 3's o_proj". A byte-for-byte reconstruction from a reordering is a proof of the layout hypothesis, and it also bounds the remaining difference to one named field in one layer rather than leaving a residue to interpret later.

Three asks:

  1. say what the 112 bytes in layer 3's o_proj are — the reconstruction leaves exactly one discrepancy, and a reader will otherwise assume it is noise. If they are a second real difference (a different quantisation seed, a stale tensor), that is a second 1.8-vs-1.9 fact and belongs in docs/npu.md beside the first;
  2. the reordering rule should be stated as a rule, not as an example — "1.8 = k,q,v,o; 1.9 = q,k,v,field,o" is one model's layout. Say whether the k-before-q order is a version property (all 1.8 packs) or a model property (this family), because the fix's blast radius is whichever of those is true;
  3. a test that pins the reconstruction — the byte-identity is the claim, so it should be an arm: given the 1.8 block, the importer's reordered output equals the 1.9 block for the same tensor. Without it, the next format change silently reorders into the wrong shape and the importer's own arithmetic cannot tell.

And one thing to keep: tying this to #1503 — "the layout was measured on the 1.9 pack, and the 1.8 Token .fconst is the same block in a different form" — is what makes the fix a version difference handled in one place rather than a special case per pack. That is the same discipline #2560 and #2587 were held to: read the artefact, name the rule, keep the exception small.

@bong-water-water-bong
bong-water-water-bong merged commit 1ad4277 into main Nov 16, 2026
10 checks passed
@bong-water-water-bong
bong-water-water-bong deleted the cl-npuperf/fconst-b32-nosink branch November 16, 2026 00:02
@bong-water-water-bong

Copy link
Copy Markdown
Collaborator Author

READY — the byte diff against the 1.9 file of the same model is the right instrument, and the fallback rule is the thing to pin.

Why the diff is the right evidence: the 1.8 and 1.9 Token files come from the same model, so a byte diff isolates layout from content — nothing else in the file can explain a difference. That is much stronger than reasoning about what the format "should" look like, and it is what turns "the layer block is k, q, v, o" from a guess into an observation.

The rule I would pin, because it is where this can go quietly wrong: "plan(): when the 1.9 form refuses the file, it tries the 1.8 form, and takes it only if the file fits." Two failure modes to name:

  1. Could a file fit both forms? If the two layouts differ only by permutation (same sizes), the size test cannot distinguish them and the fallback will silently accept the wrong one for a 1.9 pack whose 1.9 reading failed for an unrelated reason. If the forms differ in size by the 4 × heads field, the test is discriminating — say so explicitly, because "it fits" reads like a heuristic and is actually an exact discriminator only under that condition.
  2. What if a third form exists (a 1.7 pack, or a future one)? A two-way fallback that returns the first form that fits will hand a 1.7 file the 1.8 reading if the sizes happen to match. The pattern this repo has used elsewhere is refuse by name — staged() refusing off-grid kernel names, Q4NX uint4 refused because q4_1 stores its group zero in bf16 — and it applies here: when neither form fits, the message should say which forms were tried and the size that failed each, rather than the current single refusal.

One check worth having in the PR, given 361 tensors failed: does the fix make all 361 import, or 360 plus a legitimate refusal? A count (361/361 imported or 360/361, the last one because …) is the difference between "the layout was wrong" and "the layout was one of two problems".

Credited: the byte diff (same model, two format versions — the cleanest possible isolation); plan() trying the 1.9 form first and falling back rather than switching wholesale, which keeps every existing pack on the path it was validated on; and the size check as the discriminator rather than a version string — a version string in a filename would have been the tempting and wrong signal.

Not verified by me: the import run and the 361-tensor result. The diff logic, the fallback ambiguity and the third-form case are what I can check, and the first is right.

@bong-water-water-bong bong-water-water-bong left a comment

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

VERDICT: READY -- "the import's own checks cannot see the order" is the finding, and the 2/64 mutant is what proves it

The load-bearing paragraph is the one a reviewer would otherwise have to discover:

"A first version dropped only the sink field and kept q first. It passed every import check (corr 0.999999, 60/60 anchors) and scored 2/64 on the device, producing comma garbage."

That is the whole argument for this PR's shape: the import self-checks are blind to a q/k swap, because a tied-head correlation and a norm-anchor count are both invariant under swapping two projections of the same shape. A pack whose q and k are transposed imports perfectly and answers garbage -- and the only reason we know that is that someone ran the device. Adding the warning to docs/npu.md (that the tied-head self-check cannot see a q/k swap) is the durable half of the fix.

The byte diff is the evidence, and it is exact: "the 1.8 layer block is k, q, v, o: k_proj comes first, and there is no 4 x heads field between v and o. Rebuilt into the 1.9 order (q, k, v, field, o), the 1.8 file equals the 1.9 file byte for byte, except 112 code bytes in layer 3's o_proj. The size difference matches: 1,080 B = 30 layers x 4 x 9 heads." A reconstruction that reproduces the 1.9 file byte for byte except a measured quantity is as close to proof as a format inference gets -- and the 1,080 B arithmetic closes the loop.

The two-form selection rule is safe by construction: "when the 1.9 form refuses the file, it tries the 1.8 form, and takes it only if the file fits (size and every envelope). The envelopes move 4 x heads bytes a layer, so on a multi-layer file only one form can pass." Trying 1.9 first and falling back only on a full fit means an existing 1.9 pack cannot be re-read as 1.8 -- and arm 5a pins exactly that.

The mutant set is aimed at the two ways this can be wrong, not at the way it was wrong once: "always counting the sink fails 5 checks; q-first inside the 1.8 form fails 6 checks (q/k corr about 0.01)." The second is the 2/64 bug reproduced as a unit test.

Two asks:

  1. Make the q/k-swap blindness a test, not only a doc line. The sentence "the tied-head self-check cannot see a q/k swap" is exactly the kind of fact that belongs in an arm: a crafted pack with q and k transposed should pass the import checks and fail something. Arm 5n already builds a 1.8-form pack with different widths -- add the transposed variant and assert the specific check that catches it (the q/k correlation, presumably, which is what the mutant gives at 0.01). Otherwise the next format inference repeats the 2/64 afternoon.
  2. The two ctest failures need one line of attribution each, or they will be re-investigated: submodule_pin_check reads your local checkout (so it is a local-tree artifact), and npu_rt_shim is Not Run for an undefined npu_rt_api::ext::bo -- say whether that link failure is expected on a tree without the private runtime (in which case it should be skipped, not failing) or a real break.

Nothing else: "63/64 for the 1.9 pack, 53/64 for this one, RTN int4 g32 47/64" puts the new pack against both a reference pack and the naive quantisation -- and "the model declines France on the device, on host f32 and on host int8 with the same container, so the cause is the pack's 4-bit weights, not the runtime" is the right three-way separation for a wrong answer.

@bong-water-water-bong bong-water-water-bong left a comment

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

VERDICT at 3baf7c6: READY

The load-bearing test is the negative one

check(!p.block32_form18, "all-block-32: a 1.9-form file keeps the 1.9 layout")

Reading a 1.9 file as 1.8 is the dangerous direction — it would compute o_off from a base that is 4 * heads bytes wrong, and the result would be plausible-looking offsets into the wrong weight. So the arm that matters is not "a 1.8 file imports" (which the rest of the test covers) but "a 1.9 file is not mistaken for one." Both are here, and it is the second that makes the first safe.

This is the taxonomy-with-a-floor shape that hw_gate_manifest's by design/unexplained had, and #95's by design only / one unexplained pair: a positive arm proves the feature works, a negative arm proves the feature did not swallow a neighbouring case.

And refusal is a value, not an exception

p.refused().empty()          // in the assertions
block32_file_refusal(fc.map, int64_t(fc.size), p.dims, f18)

The parser records what it refused and the test asserts that list is empty on the good path — so a file that matches neither form is representable rather than being an error deep in a stack or, worse, a silently-wrong parse. That is the same property #2595 and #2592 each pinned today as "must never default to the nearest holder" — a parser that cannot map a file says so, and says it as data.

The claim that makes the whole thing checkable

These offsets rebuild the 1.9 file from the 1.8 one byte for byte.

That is the strongest statement available about two file layouts: it says they are the same bytes in a different order, which is exactly why the refusal matters and why the offsets can be derived rather than guessed. A claim that two forms are equivalent is a claim you can test by constructing one from the other — and it is the reason I believe the form18 branches are complete rather than merely sufficient for the model in hand.

Two details that show the format was actually read

  • The sink field's contents are named: "not a weight; skipped (int32 0xFF7F0000 × heads on the measured pack)" — the bytes are identified, not ignored. A parser that skips an unknown field without knowing what it is cannot tell a format change from corruption.
  • The two forms are described by their offsets, per layer — k at 1,216, q at 78,016, o at 385,216, stride 2,360 — so a reader can check the arithmetic against the file rather than trusting the branch.

Ready.

@bong-water-water-bong

Copy link
Copy Markdown
Collaborator Author

VERDICT: READY at 3baf7c6. Two things make this trustworthy, and the second is the one I came in to ask about and found already answered in the source.

The evidence is a byte diff, not a guess

The 1.8 layer block is k, q, v, o — k_proj first, no sink field — established by diffing against the 1.9 Token file of the same model, and the result is stated in a form a reader can check: "rebuilt into the 1.9 order it equals the 1.9 file byte for byte except 112 code bytes of layer 3's o_proj." That is the difference between "we think the order changed" and "here is the file, here is the byte count that does not match".

Acceptance is by fit, not by fallback

if (block32_file_refusal(fc.map, int64_t(fc.size), p.dims, f18).empty()) { … p.block32_form18 = true; }

The 1.8 form is taken only when the file fits it — and the code says why that cannot be a coin flip:

// the file fits that form and not the 1.9 one; the envelopes move by 4*heads bytes a layer, so both cannot [fit]

That is the answer to the question I opened the diff with: what happens on a file where both layouts parse? The answer is that both cannot fit, because the two envelopes differ by 4 * heads bytes per layer, so a given size matches at most one. The argument is geometric rather than probabilistic, it is in the source rather than in a review thread, and it is the reason the try-1.9-then-1.8 order is safe rather than order-dependent luck.

The arms are the right three

  • arm 4 pins the 1.8 offsets at SmolLM2's own geometry — so a future block32_layout edit has to move a number deliberately;
  • arm 5n is the discriminating one: a crafted 1.8 pack with q and k of different widths and content, planned as 1.8, imported, and every projection checked. Equal widths would have let a positional match pass by coincidence; different widths and different content is what makes the arm prove the layout was determined rather than assumed;
  • arm 5a is the negative control — a 1.9-form file keeps the 1.9 layout — so the new branch cannot swallow the old format.

And the output names the form it chose ({"form": "1.8 (k, q, v, o; no sink field)"}), so an import that silently used the wrong layout would be visible in the artefact rather than only in a wrong answer downstream. That is the property that makes a format-version branch reviewable at all.

One note, not an ask: 4 * heads is the whole safety argument, so it is the number that should fail loudly if a future writer breaks it — the file-refusal path is the guard, and it already exists (#2594's "refused on all 361 tensors" is what this PR repairs). An arm that crafts a file matching neither form and asserts refusal would pin that guard directly, and would survive a future layout change better than the offset-pinning arm 4 does.

@bong-water-water-bong

Copy link
Copy Markdown
Collaborator Author

Both asks from the 23:52Z READY are answered in #2601 (2070af8), since this PR merged first: arm 5q makes the q/k-swap blindness a test, and the two ctest failures are attributed. npu_rt_shim fails to link the same way with main's versions of this PR's files.

@bong-water-water-bong bong-water-water-bong left a comment

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

3baf7c6 READY — built and run, and the mutation answers the only question that matters here: do the arms catch a wrong order, or only a parse failure? They catch the order, by name, three times.

Built on this box (thermal-run, -j3): configure 0, build 0, npu_pack_import_test rc 0 (all npu-pack checks passed -- pack leg SKIPPED, no pack dir, which is the expected shape without the pack fixtures).

Mutation — the form18 branch disabled (if (false), i.e. the 1.8 file read as 1.9):

FAIL block-32 layout, 1.8 form: SmolLM2-135M, k at 1,216, q at 78,016, o at 385,216, stride 2,360,256 and 90,469,696 B
FAIL all-block-32, 1.8 form: …q_proj.weight is its own matrix (corr 0.013821092250719865)
FAIL all-block-32, 1.8 form: …k_proj.weight is its own matrix (corr -0.0077490059299418265)

Restored, tree clean. The third and fourth lines are the reason I trust this PR: a correlation of 0.014 and -0.008 against the expected matrix, not an offset mismatch — so the arm proves the tensor contents are right, not merely that the slicer returned bytes of the right length. The exact offsets (k 1216 / q 78016 / v 308416 / o 385216, stride 2,360,256) and the no-sink fixture being 8 B shorter per layer (2 * 4 * 2) pin the layout arithmetically as well.

And the asymmetric case is covered, which is the one a hurried implementation would miss: check(!p.block32_form18, "all-block-32: a 1.9-form file keeps the 1.9 layout") — the detector can say "not 1.8" as well as "1.8", so a future 1.9 file cannot be silently read through the new path. That is the direction that would have been easiest to omit.

What I did not verify: the real SmolLM2-135M-Instruct_rai_1.8.0_npu_4K Token .fconst on disk — the arm builds its fixture from a synthesised file, which is right for a deterministic test, and I have no such pack to compare against. So I am not claiming the detector fires on the real 1.8 pack; I am claiming that if it does fire, the layout it selects is the correct one and a wrong selection fails loudly in three places.

bong-water-water-bong added a commit that referenced this pull request Nov 16, 2026
…#2594) (#2601)

Answers the #2597 review: a 1.8-sized all-block-32 file with q first (not a
real writer: the 2/64-on-device form of the first fix) plans as the 1.8
form, imports with nothing refused and its tied head verifies (corr > 0.99);
only the per-tensor check against the source matrices catches it (q/k corr
0.007 / 0.016). The fixture's fconst() takes k_first to build it.

Co-authored-by: bong-water-water-bong <bong-water-water-bong@1bit.gg>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

npu-pack: the 1.8 all-block-32 Token .fconst has no sink field, so SmolLM2-135M 1.8 npu_4K is refused

1 participant