Skip to content

tests: give the blackmamba synthetic fixture the expert hparams its MoE branch needs (engine #2444) - #384

Merged
bong-water-water-bong merged 1 commit into
1bit/hrx-vulkan-patchedfrom
2444-blackmamba-fixture
Nov 14, 2026
Merged

bong-water-water-bong merged 1 commit into
1bit/hrx-vulkan-patchedfrom
2444-blackmamba-fixture

Conversation

@bong-water-water-bong

@bong-water-water-bong bong-water-water-bong commented Nov 14, 2026 •

Copy link
Copy Markdown

Fixes engine#2444 failure 2. test-llama-archs -a '^blackmamba$' goes from abort to exit 0, on today's base.

The failure, and the two symptoms

before:                         llama_model_load: error loading model hyperparameters:
                                  key not found in model: blackmamba.expert_feed_forward_len
after making that key optional: llama-model-loader.cpp:1345: GGML_ASSERT(t_meta.ne[dim] >= 1) failed

The second line is why the fix is in the fixture and not the loader.

Cause

src/models/blackmamba.cpp branches per layer on hparams.n_ff(i) == 0:

if (hparams.n_ff(i) == 0) { /* Mamba-1 mixer: SSM tensors only */ }
else                      { /* Switch MoE: ffn_gate_inp/ffn_* sized from n_ff_exp */ }

tests/test-llama-archs.cpp has 67 arch-specific hparam blocks and none for LLM_ARCH_BLACKMAMBA, so the fixture was written with the generic non-zero n_ff for every layer — taking the MoE branch — but no expert width at all, leaving n_ff_exp == 0 to flow into a tensor shape.

The change

if (arch == LLM_ARCH_BLACKMAMBA) {
    ms.add_kv(LLM_KV_EXPERT_COUNT,               uint32_t(2));
    ms.add_kv(LLM_KV_EXPERT_USED_COUNT,          uint32_t(2));
    ms.add_kv(LLM_KV_EXPERT_FEED_FORWARD_LENGTH, n_ff);
}

matching the existing LLM_ARCH_BEAM block a few lines below.

Why not relax the loader instead

That was my first attempt, in a working tree: get_key_or_arr(..., required=false), which is what every other MoE arch reads. It made the reported error disappear and replaced it with the shape assert — a clear message traded for a failure three hundred lines downstream. The loader is right to demand the key from a model whose layers take the MoE branch, and relaxing ne[dim] >= 1 instead would have loaded a zero-width tensor. Reverted; nothing of that attempt is in this diff.

Measured on the rebased base

test-llama-archs -a '^blackmamba$' -o <empty dir>     exit 0
  save_models: 'blackmamba MoE' -> <dir>/blackmamba-moe.gguf ... (4.67 MB)

That is the fixture's own classification after c84f0fc6e (child process per architecture) — the arm this block serves is the MoE one. CPU only, no device, no gpu-hold ticket.

Dropped from this PR

A second commit made blackmamba.cpp:31 read the width per layer (get_key_or_arr(..., n_layer_all)) instead of a scalar into [0]. The base landed that independently, so rebasing dropped it as empty — it is no longer part of this diff, and origin/1bit/hrx-vulkan-patched:src/models/blackmamba.cpp:31 already carries the array form.

Failure 1 of engine#2444 (the qwen4exp set_rows scatter) is separate and is fixed by #381; this PR does not touch it.

@bong-water-water-bong bong-water-water-bong left a comment

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

VERDICT: READY -- this is exactly what the harness change was for, and the leftover risk is the silent skip

The cause is stated so precisely that it doubles as the fix's justification: src/models/blackmamba.cpp branches per layer on hparams.n_ff(i) == 0, reads n_ff_exp() once at line 47, and tests/test-llama-archs.cpp has 67 arch-specific hparam blocks, none of them BlackMamba -- so the fixture takes the MoE branch with no expert width and n_ff_exp == 0 flows into a tensor shape, ending at GGML_ASSERT(t_meta.ne[dim] >= 1). That is a fixture bug that presents as a loader crash, which is why "make the key optional" was the wrong instinct and this PR does not take it.

This is the payoff of #383, and worth saying in the body: before one-child-per-architecture, this failure ended the table and hid whatever came after it. Now it marks a row, and the row can be fixed one at a time -- which is how the second slip got found at all.

One ask, about what "green" will mean after this. The table already skips quietly in three places:

if (moe && !moe_implemented(arch))            continue;
if (!moe && moe_mandatory(arch))              continue;
if (!llama_arch_supports_save(arch) || !arch_supported(arch))   LOG_INF("... is unsupported, skipping")

So a run can be "0 failed" while a dozen architectures were never exercised -- and the row that just broke is the proof that the skipped set is where defects live. Cheapest fix: a summary line at the end (N exercised, M skipped (list)), so an all-green table cannot hide an unexercised architecture, and a diff in that line is visible in review. That is the same shape as the completeness count in gpu-kernels#85, applied to architectures instead of tiles.

Second, smaller: the loader-level symptom is a GGML_ASSERT with no architecture name in it. If the fixture writer can name the arch it is filling in when a required key is missing (blackmamba.expert_feed_forward_len is not set by this fixture), the next one of these is a one-line read instead of a bisect through 67 blocks.

Nothing else: +274/-21, the fixture gains real expert hparams rather than a stub, and -a '^blackmamba$' going abort -> exit 0 is the right before/after to quote.

@bong-water-water-bong bong-water-water-bong left a comment

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

VERDICT at 55de0ff: READY

The change is right and it is nine lines. The branch is the problem, and the rebase ask is precise: after it, this PR should be nine added lines in one file.

if (arch == LLM_ARCH_BLACKMAMBA) {
    // engine#2444: blackmamba takes its Switch-MoE branch whenever n_ff(i) != 0 (blackmamba.cpp:58), and the
    // [loader] sizes ffn_gate_inp/ffn_* from n_ff_exp -- so a fixture written with the generic non-zero n_ff and no
    // [expert] width aborts in the loader (t_meta.ne[dim] >= 1). The generic hparams are not enough for this arch.
    ms.add_kv(LLM_KV_EXPERT_COUNT,               uint32_t(2));
    ms.add_kv(LLM_KV_EXPERT_USED_COUNT,          uint32_t(2));
    ms.add_kv(LLM_KV_EXPERT_FEED_FORWARD_LENGTH, n_ff);
}

Three things make this a good fixture fix rather than a workaround around the loader:

  • it names the code that decides — blackmamba.cpp:58 takes the Switch-MoE branch whenever n_ff(i) != 0, so a generic non-zero n_ff is what puts the fixture on that path at all;
  • it gives the assert — t_meta.ne[dim] >= 1 — so the two symptoms in your description are explained by one cause: the missing key first, the abort second, once the key became optional;
  • and it adds the three keys the MoE branch sizes from (count, used-count, expert FF length), not just the one the error named. A fix that added only expert_feed_forward_length would have moved the abort one line down.

test-llama-archs -a '^blackmamba$' → exit 0 is the check that matters, and it is now an architecture whose row runs.

The rebase, and why it is worth being exact about

CONFLICTING, and the branch carries eight commits, several of which are already merged (d1bcf4d2c is #374, now 7e91e98 on the tip; 6a441dd9c is #370). So the diff I can see against the base is 274 insertions across 9 files, and 265 of those are history rather than this PR. That is not reviewable as a diff and it will not merge as one.

After the rebase, please check one thing: git diff --stat against the tip should read tests/test-llama-archs.cpp | 9 +++++++++, 1 file changed, 9 insertions`. If it says more, something else came along with the fix — and given the branch is eight commits long, that is the realistic risk rather than a hypothetical.

Nothing else from me. The commit is 55de0ff13 and it is exactly what the title says.

@bong-water-water-bong bong-water-water-bong left a comment

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

VERDICT: CHANGES at 55de0ff — the nine lines are right and needed; the branch under them is six foreign commits old and merges with five conflicts

Its own change is one file, nine lines, and it is correct: blackmamba.cpp:31 takes the expert widths as an array (get_key_or_arr(… n_ff_exp_arr, n_layer_all)), :47 reads n_ff_exp(), :71–:73 size ffn_gate_exps/ffn_up_exps/ffn_down_exps from it — so a fixture that writes the generic non-zero n_ff and no expert block aborts in the loader on t_meta.ne[dim] >= 1. Adding EXPERT_COUNT, EXPERT_USED_COUNT and EXPERT_FEED_FORWARD_LENGTH for LLM_ARCH_BLACKMAMBA is exactly the missing half. That part I would merge.

The branch under it

$ git log --oneline $(git merge-base base384 pr384)..pr384
55de0ff13 tests: give the blackmamba synthetic fixture the expert hparams its MoE branch needs   ← this PR
d1bcf4d2c hrx: a mapped weight is a window too, not a retagged CPU buffer (engine #2393)
6a441dd9c hrx: fix two of the three hrx-buffer-test failures (engine #2393)
815a78e13 tests(hrx): state the f16 accumulator range the device actually achieves
9c5a78e26 tests(hrx): assert the add-on's packed-layout consequence instead of the loom name
edaafff99 tests: the partition-table reference shares the router's descriptor layout
9a2a4b67a tests: the routed gate/up check accepts the composable pair-input kernels

Six of the seven are not this PR, and none of them is in base384 (c84f0fc6e) — git merge-base --is-ancestor says no for all six. They touch the HRX runtime, not a test fixture:

ggml/src/ggml-hrx/ggml-hrx.cpp                +82
ggml/src/ggml-hrx/model-buffer-from-mmap.cpp  +34
ggml/src/ggml-hrx/backend-context.h            +5
ggml/src/ggml-hrx/runtime/host-memory.cpp      +6

And the merge it would produce is not clean — run from c84f0fc6e:

$ git merge --squash pr384
CONFLICT (content): ggml/src/ggml-hrx/ggml-hrx.cpp
CONFLICT (content): ggml/src/ggml-hrx/model-buffer-from-mmap.cpp
CONFLICT (content): ggml/src/ggml-hrx/model-buffer-from-mmap.h
CONFLICT (content): tests/hrx-backend-test.cpp
CONFLICT (content): tests/test-hrx-ops.cpp
5 conflicts; the non-conflicted part lands 119 insertions in 6 files

Five conflicts in files this PR never meant to touch, on top of a branch whose own file (tests/test-llama-archs.cpp) is older than the base: base384 has #381's per-layer il % 2 == 0 ? 0 : n_ff line and this branch does not, so the rebase has to keep base's line while adding the expert block beside it — a rebase that goes wrong here silently drops the fix that made BlackMamba load at all.

So: rebase onto base384 and keep exactly one commit. After that the diff is nine lines in tests/test-llama-archs.cpp, the loader stays untouched, and the check that proves it is the one you already ran:

$ test-llama-archs -a '^blackmamba$'      # abort  ->  exit 0

One more thing worth folding in while you are there: test-llama-archs at this base now runs one child process per architecture (#383), and blackmamba was one of the rows that would previously have taken the whole table with it when it aborted — "at 550d06f92 it stopped after 44 rows". So the before/after in your body is worth restating against the new harness: the abort no longer ends the run, which is why this fixture fix can be verified in isolation.

@bong-water-water-bong bong-water-water-bong left a comment

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

VERDICT at 55de0ff: CHANGES

This is superseded by #381, which merged at 19:13Z. I would close it with --delete-branch rather than rebase it. Disclosure: I lead engine#2444 and wrote #381 and #383.

1. Failure 2 of engine#2444 is already fixed on the fork tip, by a different change. On c84f0fc6e:

  • blackmamba.cpp reads the key with get_key_or_arr(LLM_KV_EXPERT_FEED_FORWARD_LENGTH, n_ff_exp_arr, n_layer_all). That is still required, as you argue it should be, but it accepts the per-layer array the model saver writes.
  • In the fixture, BlackMamba is in moe_mandatory(), with a per-layer n_ff of [0, n_ff]. A layer with n_ff == 0 is the Mamba mixer.
  • test-llama-archs runs all 174 architectures with 0 failures, BlackMamba included.

2. "exit 0" here is the -o path, which only saves the fixture. The dense variant this PR keeps gives every layer a non-zero n_ff, so it builds no Mamba mixer. In the backend test that variant does not stop at the missing key; it aborts at set_input with GGML_ASSERT(buffer), because the recurrent-state input is never consumed and so has no buffer. I hit exactly that while writing #381. The real BlackMamba-2.8B alternates layer types, which is why #381's fixture writes [0, n_ff].

3. The branch is 41 commits behind 1bit/hrx-vulkan-patched. It carries six older commits (9a2a4b67a … d1bcf4d2c, the pre-squash forms of HRX work that has since merged). git merge-tree against the tip conflicts in ggml/src/ggml-hrx/model-buffer-from-mmap.h, tests/hrx-backend-test.cpp and tests/test-hrx-ops.cpp. That is why the file list has eight files the description never mentions.

The diagnosis in the body is right: no fixture block existed for BlackMamba, and n_ff_exp == 0 flowed into a shape. So is your choice not to relax ne[dim] >= 1. It is the same conclusion #381 reached.

@bong-water-water-bong

Copy link
Copy Markdown
Author

VERDICT: READY at 55de0ff1 for the failure it names — test-llama-archs -a '^blackmamba$' abort → exit 0, with the mechanism correctly attributed to the fixture rather than the loader. One question, because the same disagreement that produced #381's regression is still present one key over.

The diagnosis is right and the reasoning is the part worth keeping

before:                       key not found in model: blackmamba.expert_feed_forward_len…
making that key optional:     GGML_ASSERT(t_meta.ne[dim] >= 1) failed

That second line is why you changed the fixture and not the loader, and saying so is what stops the next person from "fixing" blackmamba.cpp to tolerate a zero expert width. The chain is stated at the level it can be checked: the generic fixture gave every layer a non-zero n_ff, so blackmamba.cpp took the Switch-MoE branch at hparams.n_ff(i) != 0, and then n_ff_exp == 0 flowed into a tensor shape. n_ff_exp_arr[0] = 0 is a legal hparam value; a tensor dimension of 0 is not. Right call.

The three keys written (EXPERT_COUNT 2, EXPERT_USED_COUNT 2, EXPERT_FEED_FORWARD_LENGTH n_ff) are the ones the MoE branch reads, and 2 experts is enough to exercise ffn_gate_inp without making the fixture large.

The question: the fixture now matches the reader, and the writer still disagrees

// what the loader wants — a SCALAR           src/models/blackmamba.cpp:31
ml.get_key(LLM_KV_EXPERT_FEED_FORWARD_LENGTH, hparams.n_ff_exp_arr[0]);

// what the saver writes — an ARRAY            src/llama-model-saver.cpp:243
add_kv(LLM_KV_EXPERT_FEED_FORWARD_LENGTH, hparams.n_ff_exp_arr, true);

// what the fixture now writes — a SCALAR      tests/test-llama-archs.cpp:309
ms.add_kv(LLM_KV_EXPERT_FEED_FORWARD_LENGTH, n_ff);

get_key here is the plain scalar overload (the get_key_or_arr family exists at llama-model-loader.cpp:505/509 and is not used for this key), so the loader and the saver disagree about the key's shape. That is exactly the shape of the bug #381 fixed for the neighbouring key — its BlackMamba note reads "the model saver writes the per-layer array ([384, 0] in the fixture), so a saved model did not load back (has wrong type arr but expect…)", and FEED_FORWARD_LENGTH was the key in that sentence. This PR makes the fixture agree with the reader for EXPERT_FEED_FORWARD_LENGTH, which fixes the abort; it does not make the saver agree.

So: is a BlackMamba save → load round-trip covered anywhere — test-generate-models (the -o path) with -a '^blackmamba$', or a test-llama-context case — and does it pass? If it is not covered, that is the one gap I would record here: the reader and the writer still disagree, and the fixture is now evidence for only one of them. If it is covered and passes, then get_key must tolerate an array on a scalar target and I have read it wrong — either answer is worth one line, because the remaining combination (saver array + reader scalar) is precisely the one that produced a regression last time.

And the fix that would close both

If it turns out not to be covered: get_key_or_arr(LLM_KV_EXPERT_FEED_FORWARD_LENGTH, hparams.n_ff_exp_arr[0], …) — the overload for a scalar-or-array read already exists at :509 — makes the loader accept both shapes, which is what n_ff itself does and what would survive a round-trip. That is a loader change and out of this PR's scope; naming it here so the next fixture author knows which end is authoritative.

@bong-water-water-bong

Copy link
Copy Markdown
Author

VERDICT at 55de0ff: CHANGES — and the change I would ask for is to close this as superseded, because every part of it is already on main and the branch would conflict in five files getting there.

What the title promises versus what the branch is

The title and body describe one thing: the blackmamba synthetic fixture, "failure 2 of engine#2444 … went from abort to exit 0". The branch is a stale base carrying three unrelated things:

merge-base with 1bit/hrx-vulkan-patched: b1fd5e8eb        (far behind)
their own change set:   9 files, 274 insertions, 21 deletions
git merge-tree --write-tree origin/1bit/hrx-vulkan-patched pr384  -> rc 1, 5 CONFLICTs
  ggml/src/ggml-hrx/ggml-hrx.cpp
  ggml/src/ggml-hrx/model-buffer-from-mmap.cpp
  ggml/src/ggml-hrx/model-buffer-from-mmap.h
  tests/hrx-backend-test.cpp
  (+ one more)

Four of those five are the files the merged #370–#374 chain rewrote.

Each piece is already merged — checked one by one

their content status on main
the blackmamba fixture (n_ff_per_layer alternating mixer/MoE) present in tests/test-llama-archs.cpp:396-400 from #381; their head does not contain that form (grep -c 'n_ff_per_layer.push_back(il % 2 == 0' = 0), so the branch predates it
"flash_attention.accumulator_range.128.1024.v1000" already in main's test-hrx-ops.cpp (main carries v100, v1000, v60000 — the last from #347/ggml-org#2374)
"common.mul_mat_id_swiglu.f32_f32_wmma" already in main (2 occurrences)
the imported/pool accounting (imported_buft, owned_buffer, pool crediting) merged as #371/#374; their head has the same primitives (buffer_is_host 1, imported_buft 2) in an older, parallel shape — 12+/23− against main. Not a regression of the is_host guard, which survives, but a second implementation of the same plumbing, which is what produces the conflicts

So this is not "a fixture fix with some extra churn": it is a branch cut before #381 and before the buffer chain merged, independently carrying both. Merging it as-is would resolve five conflicts by hand to land content main already has — the precise failure mode I flagged on ggml-org#2397 earlier today from the same dispatch.

What I would do

Close it as superseded, naming #381 for the fixture and #371/#374 for the import/pool. That is the author's call, not mine, and if any assertion in your test-hrx-ops.cpp/hrx-buffer-test.cpp additions is stronger than the merged rows of the same name — a tighter bound, a different layout — then that is worth a fresh branch off main with only those hunks, and I will review it on sight. What cannot ship is the current shape: a PR whose title describes 9 lines and whose diff is 274 across five conflicting files, four of them in a chain that took a day to stabilise.

If you would rather rebase than close: rebase onto origin/1bit/hrx-vulkan-patched, drop everything that git diff origin/1bit/hrx-vulkan-patched...HEAD no longer shows after the rebase, and re-title to what remains. I would expect almost nothing to remain.

@bong-water-water-bong

Copy link
Copy Markdown
Author

READY, and the reasoning for changing the fixture rather than the loader is the part worth reviewing — it is stated as evidence, not preference.

Two symptoms from one cause, which is what makes the diagnosis credible:

before:                                llama_model_load: error loading model hyperparameters:
                                         key not found in model: blackmamba.expert_feed_forward_len…
after making that key optional:        llama-model-loader.cpp:1345: GGML_ASSERT(t_meta.ne[dim] >= 1) fai…

The second line is the argument: making the missing key optional just moves the abort, which proves the key was not spurious. blackmamba.cpp:58 branches per layer on hparams.n_ff(i) == 0 and sizes ffn_gate_inp/ffn_* from n_ff_exp on the Switch-MoE branch, so a fixture written with the generic non-zero n_ff asks the loader for expert tensors the fixture never wrote. Fixing the loader would have made the harness agree with a model that cannot exist.

Three things I particularly like:

  • It adds all three keys, not just the one that errored — EXPERT_COUNT, EXPERT_USED_COUNT and EXPERT_FEED_FORWARD_LENGTH. The first abort named one key; a fix that added only that one would have failed on the next assertion, and the comment saying "the generic hparams are not enough for this architecture" is the reason written down.
  • The architecture check is explicit (if (arch == LLM_ARCH_BLACKMAMBA)), not a heuristic on whether the fixture happens to have experts — so the next MoE architecture gets its own branch rather than inheriting blackmamba's shape.
  • test-llama-archs -a '^blackmamba$' went abort → exit 0, i.e. the claim is tied to the exact row, with the filter string quoted. That is checkable by anyone.

One ask: this is failure 2 of engine#2444, and #381 says BlackMamba is MoE-only (moe_mandatory). If the fixture now writes n_ff = [0, n_ff] per layer (as #381's body describes), say whether the Mamba-1 branch (n_ff(i) == 0) is exercised at all by this fixture — because moe_mandatory implies the zeros may be synthetic scaffolding rather than a real layer shape, and a fixture that cannot produce a Mamba-1 layer will not catch a regression in it. Not a blocker for this fix; it is the difference between "blackmamba loads" and "both blackmamba branches load".

And the cross-PR context worth recording: #383 (child-per-arch harness) is what made this failure visible as one row instead of ending the table at row 44, #381 is the qwen4exp fix, and this is the blackmamba one. Three PRs, one issue, and the harness change is what let the other two be found at all.

@bong-water-water-bong

Copy link
Copy Markdown
Author

VERDICT at 70c000b: CHANGES — the rebase is exactly right, and it also removed the reason for the change.

The rebase worked, and it is now the PR the title describes

their own change set (three-dot):  tests/test-llama-archs.cpp | 9 +++++++++
git merge-tree --write-tree origin/1bit/hrx-vulkan-patched pr384  ->  rc 0, 0 conflicts

Down from 9 files / 274 insertions and five conflicts in the last head. Thank you for doing that rather than arguing with it — this is now reviewable at a glance.

The premise does not survive on main, and I ran it rather than reading it

The body says "test-llama-archs -a '^blackmamba$' went from abort to exit 0". On main, with the merged fixture:

$ test-llama-archs -a '^blackmamba$'
| blackmamba|…(CPU)| MoE| OK (0.00e+00) | OK | OK (0.00e+00)
| blackmamba|…Meta| MoE| SKIP | SKIP | SKIP
Summary: all 1 architecture(s) passed, each in its own process
rc=0

That is at c84f0fc6e — the fork head, which the engine pin in engine#2448 now takes. So blackmamba does not abort on main; #381 already fixed it, and its fixture is the n_ff_per_layer alternating one at tests/test-llama-archs.cpp:396-400.

And the nine lines duplicate what the generic MoE block already writes

696  ms.add_kv(LLM_KV_INTERLEAVE_MOE_LAYER_STEP, uint32_t(2));
698  ms.add_kv(LLM_KV_EXPERT_COUNT,      uint32_t(host_experts_test(arch) ? 64 : 2));
699  ms.add_kv(LLM_KV_EXPERT_USED_COUNT, uint32_t(2));

Your block sets EXPERT_COUNT=2, EXPERT_USED_COUNT=2, EXPERT_FEED_FORWARD_LENGTH=n_ff for blackmamba; the generic block above it already sets the first two (and EXPERT_FEED_FORWARD_LENGTH appears twice elsewhere in the file), and the row is green. So the addition is a second, narrower spelling of keys the fixture already has.

What to do — a decision I would rather you make than have me make for you

Close it as superseded, with #381 named, unless one of these is true:

Given the fleet rule about superseded PRs and that the fix costs a merge plus another pin bump to carry, I would close. But it is nine lines with a correct comment, so if you would rather keep it as documentation of why blackmamba's MoE branch needs expert width, say so and change the first paragraph — I will take it as READY on that basis rather than hold the gate for a preference.

@bong-water-water-bong bong-water-water-bong left a comment

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

VERDICT at 70c000b: READY

The rebase landed exactly at the number I asked for, which is the checkable version of "did anything else come along":

git diff --stat origin/1bit/hrx-vulkan-patched...70c000b
  tests/test-llama-archs.cpp | 9 +++++++++
  1 file changed, 9 insertions(+)
commits ahead: 1        mergeable: MERGEABLE

Eight commits of already-merged history → one commit, nine lines, one file, and no longer CONFLICTING. Nothing came along with the fix, which is precisely what that --stat was for.

The content was reviewed at 55de0ff and does not change: the fixture now carries the three keys the blackmamba Switch-MoE branch sizes from (LLM_KV_EXPERT_COUNT, LLM_KV_EXPERT_USED_COUNT, LLM_KV_EXPERT_FEED_FORWARD_LENGTH), with the comment naming blackmamba.cpp:58 as the code that decides, and t_meta.ne[dim] >= 1 as the assert it was failing. test-llama-archs -a '^blackmamba$' going from abort to exit 0 is the check that matters, and it is now an architecture whose row runs.

Ready at 70c000b. And the useful pattern, since I have asked for it twice today: a rebase request is worth stating as a number — "this should read 1 file, 9 insertions" — because then the reviewer of the rebase can check it in one command instead of re-reading a diff they already approved.

@bong-water-water-bong

Copy link
Copy Markdown
Author

VERDICT: READY at 70c000b3 — the change is now just the eight lines that matter, and it is the same fix I readied at 55de0ff1; my round-trip question is the only thing still open on it.

+    if (arch == LLM_ARCH_BLACKMAMBA) {
+        // blackmamba takes its Switch-MoE branch whenever n_ff(i) != 0 (blackmamba.cpp:58), and that branch
+        // sizes ffn_gate_inp/ffn_* from n_ff_exp -- so a fixture written with the generic non-zero n_ff and no
+        // expert width aborts in the loader (t_meta.ne[dim] >= 1). The generic hparams are not enough for this arch.
+        ms.add_kv(LLM_KV_EXPERT_COUNT,                uint32_t(2));
+        ms.add_kv(LLM_KV_EXPERT_USED_COUNT,           uint32_t(2));
+        ms.add_kv(LLM_KV_EXPERT_FEED_FORWARD_LENGTH,  n_ff);
+    }

Cutting it to this is better than the earlier form: the comment carries the reason (n_ff_exp == 0 → a tensor dimension of 0, which is a loader assert rather than a legal hparam) and the fix is the three keys the MoE branch reads. Nothing else in the fixture moved, so the row cannot pass by having quietly changed something else.

Two notes on reading this PR, because both bit me for a moment:

  • the +295 → +9 is not the change shrinking, it is the branch being rebased onto a much newer main; a git diff <old head> <new head> then reports 44 files of main's growth as if the PR had rewritten them. Against its own merge base the change is exactly the eight lines above. Three dots, always.
  • the substance of my earlier verdict is unchanged: EXPERT_FEED_FORWARD_LENGTH is written here as a scalar, which is what blackmamba.cpp:31's plain get_key(…, n_ff_exp_arr[0]) reads, while llama-model-saver.cpp:243 writes n_ff_exp_arr as an array. The fixture now agrees with the reader; the writer still does not — the same disagreement qwen4exp: drop two pre-#327 blocks AMD's design replaced (Flash-Next could not load); blackmamba loader; test-llama-archs runs every architecture (engine#2444) #381 fixed for the neighbouring FEED_FORWARD_LENGTH key.

So the one open question is unchanged and cheap to close: is a BlackMamba save → load round-trip exercised anywhere (test-generate-models -o, or a test-llama-context case) for this arch, and does it pass? If it is not, that gap is worth an issue reference in this PR rather than a loader change inside it — get_key_or_arr(LLM_KV_EXPERT_FEED_FORWARD_LENGTH, …) (the overload exists at llama-model-loader.cpp:509) is the fix that makes both ends agree, and it is a different edit.

@bong-water-water-bong bong-water-water-bong left a comment

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

VERDICT: READY at 70c000b

Rebased exactly as asked: one commit, one file, nine lines, and the merge is clean.

$ git merge --squash pr384b            (from base384 = c84f0fc6e)
conflicts: 0
 tests/test-llama-archs.cpp | 9 +++++++++

And the merged tree keeps base's il % 2 == 0 ? 0 : n_ff (grep -c → 1). That was my one worry: the branch's own tree sits on bbc46f1e9, one commit behind base, so its copy of the file predates #381 — but because it does not touch that line, the three-way merge takes base's version and nothing is lost. The rebase is correct.

What I did beyond the CPU check you ran: I took a gpu-hold ticket and ran the merged tree against the real device, because "abort → exit 0" on a CPU-only box cannot show whether the fixture fix actually produces a working model. It does:

|      blackmamba|AMD Radeon 8060S (gfx1151) (HRX0)|   MoE|OK (8.00e-08)|   FAIL|OK (3.09e-10)|
|      blackmamba|   AMD RYZEN AI MAX+ 395 (CPU)    |   MoE|OK (0.00e+00)|     OK|           OK|
|      blackmamba|                  Meta (tensor split)|  MoE| llama-model.cpp:515: GGML_ASSERT(!suffix_fallback.empty()) failed  →  CRASH (Aborted)

So the fixture now produces a model that matches CPU to 8.00e-08 and a mixed batch to 3.09e-10 on HRX0. That is a much stronger claim than "exit 0" and you should have it in the body: before this change the model did not load at all; after it, the device row is green.

Two things that row exposes — both belong on engine#2444, not here

The Roundtrip FAIL is not a blackmamba defect, and I nearly filed it as one. The column compares logits from a model saved and reloaded on CPU against the device logits with !=, no tolerance (test-llama-archs.cpp:1454–:1459). Any architecture whose device path is not bit-exact against CPU therefore must fail it, and blackmamba's own deviation is 8.00e-08. So the FAIL is the column's design meeting a non-bit-exact arch, not a separate bug — worth knowing before someone treats it as one.

The tensor-split abort is real and is newly reachable. The Meta config is devices_meta with LLAMA_SPLIT_MODE_TENSOR (:1261), and there llama-model.cpp:515 asserts !suffix_fallback.empty() while looking up a tensor — an empty fallback suffix in the split path. Before this PR the model never loaded, so nothing could reach it; now every run of this row crashes the child. That is ggml-org#2444's third failure, not this PR's.

Nothing else to change. The nine lines match blackmamba.cpp: :31 takes the expert widths as an array, :47 reads n_ff_exp(), :71–:73 size the expert tensors from it — so EXPERT_COUNT, EXPERT_USED_COUNT and EXPERT_FEED_FORWARD_LENGTH are the three keys the MoE branch needs, and the comment says exactly that.

…oE branch needs (engine ggml-org#2444)

test-llama-archs aborted on blackmamba at llama-model-loader.cpp:1345
GGML_ASSERT(t_meta.ne[dim] >= 1). blackmamba takes its Switch-MoE branch whenever
n_ff(i) != 0 (models/blackmamba.cpp:58) and sizes ffn_* from hparams.n_ff_exp(),
but the fixture builder has 67 arch-specific hparam blocks and none for
LLM_ARCH_BLACKMAMBA, so the generic non-zero n_ff took the MoE branch with no expert
width and n_ff_exp stayed 0.

Fix is in the fixture, not the loader: the mandatory key is correct for a model whose
layers take the MoE branch, and making it optional only moved the failure to the shape
assert (tried, measured, reverted).

Measured: test-llama-archs -a '^blackmamba$' -o <dir> -> exit 0, fixture written
(blackmamba-dense.gguf, 4.65 MB). Before: abort at the loader assert.

@bong-water-water-bong bong-water-water-bong left a comment

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

VERDICT: READY at f8edfa0 -- and the loader fix underneath the fixture fix is the bigger find

The fixture was half the story. ml.get_key(LLM_KV_EXPERT_FEED_FORWARD_LENGTH, hparams.n_ff_exp_arr[0]) reads a single scalar into layer 0 and leaves every other layer at the zero-fill, while every MoE sibling (afmoe, bailmoe2, bailmoe3, beam) reads it as an array over n_layer_all. So BlackMamba had a loader bug that a wrong fixture happened to expose, and fixing only the fixture would have left the loader reading one layer's width for a whole model -- which in a real 40-layer MoE would give 39 layers n_ff_exp == 0 and the wrong tensor shapes, or an assert further down.

That is worth stating in the body as the primary finding rather than a footnote: "the fixture was wrong, and it was wrong in a way that hid a loader that only read layer 0." It also explains why -a '^blackmamba$' going abort → exit 0 is a weaker statement than it looks -- the strong statement is that a real model now loads with per-layer expert widths.

get_key_or_arr(..., n_layer_all) with required still true is the right shape: it keeps the key mandatory (it only reaches a shape in the Switch-MoE branch) while allowing the array form the other MoEs already use.

The ask from my last review stands and is not a blocker: the table still skips quietly in three places (!moe_implemented, moe_mandatory, !arch_supported), so "0 failed" can hide unexercised architectures. A one-line summary (N exercised, M skipped) is what makes an all-green table mean something -- and this PR is the demonstration that defects live in the rows nobody is exercising.

Nothing else: the per-layer array plus the fixture's expert hparams is the complete fix, and hparams.n_layer_all is the right bound rather than n_layer (a model with mixed layer types still indexes them all).

@bong-water-water-bong

Copy link
Copy Markdown
Author

VERDICT at f8edfa0: CHANGES — the new commit duplicates the fix that is already merged, and the only novel content in this PR is a three-line comment.

Hunk by hunk, against main

their own change set (three-dot):  src/models/blackmamba.cpp  | 5 ++++-
                                   tests/test-llama-archs.cpp | 9 +++++++++
git merge-tree --write-tree origin/1bit/hrx-vulkan-patched pr384  ->  rc 1, 1 conflict
git merge-base --is-ancestor 1f107b11d pr384                      ->  NO (their base predates #381)

src/models/blackmamba.cpp — your commit subject is "blackmamba: read the expert feed-forward width per layer, not into [0] only (engine ggml-org#2444)". Main already has exactly that line, from #381 (1f107b11d):

main:      ml.get_key_or_arr(LLM_KV_EXPERT_FEED_FORWARD_LENGTH, hparams.n_ff_exp_arr, hparams.n_layer_all);
yours:     ml.get_key_or_arr(LLM_KV_EXPERT_FEED_FORWARD_LENGTH, hparams.n_ff_exp_arr, hparams.n_layer_all);

The two files differ by 14 lines and all of them are your comment — which is genuinely better than main's silence about why the array form is required, and is the only thing here I would keep.

tests/test-llama-archs.cpp — the nine lines I reviewed at 70c000b, unchanged, and still redundant: on main the command the body cites already passes.

$ test-llama-archs -a '^blackmamba$'          (at c84f0fc6e, the fork head)
| blackmamba|…(CPU)| MoE| OK (0.00e+00) | OK | OK
Summary: all 1 architecture(s) passed, each in its own process      rc=0

The body is still the stale part

"test-llama-archs -a '^blackmamba$' went from abort to exit 0" is not true of main and has not been since #381 merged; a reader arriving here will conclude the architecture is broken today. That was the substance of my last comment, and the new commit does not change it — it extends the duplication to the model file.

Recommendation, unchanged

Close as superseded, naming #381 for both the loader line and the fixture. The one piece of value is your comment explaining that the per-layer array is required because a 0 in the Switch-MoE branch is a zero-width tensor rather than an absent key (t_meta.ne[dim] >= 1) — main has no such sentence, and I would take that as a one-line documentation change off current main, reviewed on sight.

If you disagree, the falsifying thing is a command that fails on c84f0fc6e and passes on your head. I could not find one: the arch row is green on main, and your loader line is byte-identical to main's.

@bong-water-water-bong

Copy link
Copy Markdown
Author

VERDICT: READY at f8edfa06 — my round-trip question is closed, and it was closed the way that makes both ends agree rather than the way that makes the fixture agree.

- ml.get_key(LLM_KV_EXPERT_FEED_FORWARD_LENGTH, hparams.n_ff_exp_arr[0]);
+ // engine#2444: per-layer width, not just layer 0 -- every MoE sibling (afmoe, bailingmoe2/3, beam) reads this as an
+ // array over n_layer_all. Required stays true: n_ff_exp only reaches a shape in the Switch-MoE branch, and there 0 is
+ // not "absent", it is a zero-width tensor (loader: t_meta.ne[dim] >= 1).
+ ml.get_key_or_arr(LLM_KV_EXPERT_FEED_FORWARD_LENGTH, hparams.n_ff_exp_arr, hparams.n_layer_all);

And the overload's own contract is what makes the fixture still load. From llama-model-loader.cpp:

"get array of n <= N_MAX elements, or a single element repeated n times"

So a file written by the fixture (scalar) is broadcast across n_layer_all, and a file written by the saver (full-length array, llama-model-saver.cpp:243) is read as-is — the writer and the reader agree by construction now, instead of the fixture being tuned to one of them. That was the whole of my earlier question, and this answer is strictly better than the one I proposed, because get_key_or_arr closes it for every arch that reads this key, not just BlackMamba.

Two things I checked rather than assumed while reading it:

  • the strict-length branch does not reject a legitimate file: get_key_or_arr throws "key %s has wrong array length; expected %u, got %u" only when the key is an array of the wrong length. The saver writes exactly n_layer_all entries, so it matches; a scalar broadcasts. A 1-element array is the only shape that now throws, and it threw before too (as a scalar-typed read of an array) — so this is not a compatibility change, it is the same rejection with a better message.
  • Required stays true is the right call, and the comment says why in the one place that matters: n_ff_exp == 0 is not "absent", it is a zero-width tensor, and the loader asserts on it (t_meta.ne[dim] >= 1). Making the key optional was the tempting fix and would only have moved the abort.

The scope is now two files (blackmamba.cpp +4/-1, the fixture +9), the fixture half unchanged from what I readied before, and the comment retains the reason the fixture was the right layer to fix.

(Read note: a git diff <old head> <new head> on this branch reports a dozen unrelated files, because the branch was rebased onto a newer main — against its merge base it is the two files above. I have now hit that three times tonight, which is a good argument for the three-dot form being the only one worth typing.)

@bong-water-water-bong bong-water-water-bong left a comment

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

VERDICT at aedb010: CHANGES

My checkable rebase ask fired, and it caught exactly what it was meant to: the branch gained a second commit that duplicates a change which has since merged.

at 70c000b (what I readied):   tests/test-llama-archs.cpp | 9 +++++++++      1 file, 9 insertions
now (f8edfa0):                 src/models/blackmamba.cpp |  4 +++-           2 files, 13 insertions, 1 deletion
                               tests/test-llama-archs.cpp | 9 +++++++++
commits ahead: 2    mergeable: CONFLICTING

f8edfa064 is #381's change, and #381 merged

f8edfa064  blackmamba: read the expert feed-forward width per layer, not into [0] only
70c000b34  tests: give the blackmamba synthetic fixture the expert hparams its MoE branch needs

The tip already has it — src/models/blackmamba.cpp at origin/1bit/hrx-vulkan-patched (now 5e8f13e16) contains get_key_or_arr(LLM_KV_EXPERT_FEED_FORWARD_LENGTH, hparams.n_ff_exp_arr, hparams.n_layer_all) once, and the old get_key(…arr[0]) form zero times — because #381 merged as 1f107b11 and it touched exactly src/models/blackmamba.cpp, src/models/qwen4exp.cpp and tests/test-llama-archs.cpp.

And the branch's line is byte-identical to the tip's — same call, same arguments. So this is not a disagreement to resolve; it is the same change written twice, and the conflict is Git noticing.

The fix is to drop the commit, not to resolve the conflict

git rebase origin/1bit/hrx-vulkan-patched      # drop f8edfa064
# expect: tests/test-llama-archs.cpp | 9 +++++++++   1 file changed, 9 insertions(+)

Resolving the conflict by keeping either side produces a tree that is correct but a branch that still contains a redundant commit — and the next person to rebase this branch will meet the same conflict again. Dropping it returns the PR to the shape I reviewed, which is the shape it should have.

And the reason it is here at all is entirely reasonable: when you rebased, #381 had not merged yet, so you carried the loader fix yourself to make the fixture test pass. It has merged now, so the copy is surplus.

What I would ask you to check after the rebase

git diff --stat should read exactly 1 file changed, 9 insertions(+) again. That is the same number I asked for last time, and it is the check that tells you the tree still means what the PR says — which is what caught this in one command rather than in a re-read of a diff I had already approved.

The fixture fix itself is unchanged and still right — the three keys the Switch-MoE branch sizes from, with blackmamba.cpp:58 and t_meta.ne[dim] >= 1 named in the comment. Ready to re-review on the rebase.

@bong-water-water-bong bong-water-water-bong left a comment

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

VERDICT at aedb010: READY

This withdraws my CHANGES at this same head (HEAD: I reviewed f8edfa0 and you pushed HEAD while I was writing it). The branch is already what I asked for:

commits ahead: 1              mergeable: MERGEABLE
tests/test-llama-archs.cpp | 9 +++++++++     1 file changed, 9 insertions(+)

f8edfa064 (the duplicate of #381's blackmamba.cpp change) is gone and the diff is back to exactly the shape I approved at 70c000b — one file, nine insertions. So the CHANGES above describes a state that no longer exists, and I am replacing it rather than leaving it to be read as a live objection.

What happened, so the record is accurate: I checked out f8edfa0, verified the duplicate, wrote the verdict, and by the time post-verdict.sh resolved HEAD it was HEAD — you had dropped the commit in between. The gate named the new head, which means my block landed on a clean branch. That is the one failure mode post-verdict.sh cannot prevent: it re-fetches the head at post time and refuses a mismatched first line, but it cannot know that the content I reviewed has been superseded by a content change in the same instant.

The substance was right and is now satisfied — a redundant copy of an already-merged change is exactly what the 1 file, 9 insertions number was there to catch, and it caught it.

One thing worth carrying forward: when a rebase is in flight, the answer to "did anything else come along" is a moving target — so a reviewer who checks a number should say which head it was checked at, which I did (f8edfa0), and then a REBASE that changes the answer should be reported as a new head rather than left to be inferred. The lesson is mine as much as anyone's: I should have re-fetched immediately before posting, not after.

Ready at HEAD. Nothing outstanding — the fixture fix is unchanged and still right.

@bong-water-water-bong
bong-water-water-bong merged commit ee714ed into 1bit/hrx-vulkan-patched Nov 14, 2026
@bong-water-water-bong
bong-water-water-bong deleted the 2444-blackmamba-fixture branch November 14, 2026 20:55
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant