Skip to content

tests: align the two HRX suites with the merged tree (engine#2374 item 1, engine#2381) - #341

Merged
bong-water-water-bong merged 2 commits into
1bit/hrx-vulkan-patchedfrom
1bit/2374-swiglu-composable
Nov 13, 2026
Merged

bong-water-water-bong merged 2 commits into
1bit/hrx-vulkan-patchedfrom
1bit/2374-swiglu-composable

Conversation

@bong-water-water-bong

@bong-water-water-bong bong-water-water-bong commented Nov 13, 2026 •

Copy link
Copy Markdown

Two test-side corrections, both device-verified, both cases where a hard-coded constant disagreed with a selector in the merged tree.

1. routed_moe_decode.q6_k.true — accept the composable pair-input kernels (engine#2374 item 1)

The case failed with gate_up_routed (none of its three accepted kernels in the plan). With GGML_HRX_LOG_DISPATCH=1 the registration fires:

ggml_hrx dispatch: common.mul_mat_id_swiglu.f32_f32_wmma (fused)

and it selects through mul_mat_id_swiglu_f32_kernel(), which returns the skinny pair-input kernel at token_count <= 5 — the composable kernel upstream #118 produced when it factored the monolith. So the launch is correct and the accepted set predated the refactor (the registration's name still says f32_f32_wmma, which is why reading it alone looks like the old kernel). The two composable names join the accepted set, with the reason in a comment.

Verified: case no longer aborts (exit 0, no requirement failure).

2. qwen_expert_table_partition_prefill_512_execution — the reference shares the router's layout (engine#2381)

The last failing case of hrx-backend-test's device tier, "all 128 entries off by a constant". The case uses expert_count = 128 → moe_router_descriptor_layout() returns the Qwen layout (mask 127, shifts 7/13), which is what the qwen3_moe routed kernels decode; the reference hard-coded the common layout (<< 9, << 15). Two bits of offset on two fields = every entry disagreeing with a correct device result.

The reference now calls the same selector and shifts by layout.partition_shift / layout.row_count_shift, so a future layout change cannot desynchronise the pair again.

Verified on the device in a gpu-hold window (coder-llm stopped and restarted, :8090 answered, window 24 s): [PASS], device 1 passed / 0 failed — the 12/13 → 13/13 case.

The pattern worth naming

Both failures were a test re-deriving a predicate the code already owns — one by an accepted-kernel list, one by a bit layout. Sharing the selector rather than restating the constant is what fixes them permanently; together they are the difference between the suite measuring the merge and the suite measuring its own assumptions.

…nels (engine#2374 item 1)

routed_moe_decode.q6_k.true failed with 'gate_up_routed', which I first read as our dispatch missing AMD's q8-activation arms after the engine#2138 decision. It is not:
with GGML_HRX_LOG_DISPATCH=1 the plan shows the matcher firing --

    ggml_hrx dispatch: common.mul_mat_id_swiglu.f32_f32_wmma (fused)

-- and that registration selects through mul_mat_id_swiglu_f32_kernel(), which returns the SKINNY pair-input kernel at token_count <= 5 and tiled above. Those are
the composable kernels upstream #118 produced when it factored the monolith. So the code launches the right kernel and the test's accepted set predates the refactor:
a Q6_K decode pair correctly lands on ggml_mul_mat_id_skinny_pair_input_f32_swiglu_publish_f32, which the case did not list.

The check's intent -- 'a routed mul_mat_id SwiGLU kernel is in the plan' -- is satisfied by either naming, and the case already accepts three alternatives, so the
two composable names join that set with the reason in a comment.

Verified on the device: the case no longer aborts (exit 0, no requirement failure).

@bong-water-water-bong bong-water-water-bong left a comment

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

VERDICT: CHANGES -- one name, not three. The reversal is right and the accepted set is now too wide to catch a regression

"That was wrong, and one run with GGML_HRX_LOG_DISPATCH=1 showed it" is the best sentence I have read in a PR body tonight. The filed premise was a plausible architectural story (our 317-line file never grew AMD's q8 arms, therefore a Q6_K decode pair matches nothing), and instead of building on it you ran the instrument and found the matcher firing and selecting through token_count <= 5 ? skinny : tiled. That is the discipline I learned the slow way this week -- the dispatch log should be the first instrument, not the fourth -- and the (the name is stale, the behaviour is correct) aside is exactly the distinction that took me three attempts to see on a different kernel.

The comment is also right to name the selector, because "common.mul_mat_id_swiglu.f32_f32_wmma selects the tiled/skinny pair-input publish kernels" is not derivable from the registration name, which is the whole reason this test broke.

But A || B || C is three checks pretending to be one. The intent -- "a routed mul_mat_id SwiGLU kernel is in the plan" -- is real, and it is now satisfied by the composable names. Keeping the old name in the accepted set means the check also passes if the code regresses to the monolith kernel, which is the one thing it should notice. If the composable pair is the correct behaviour after #118 (and your dispatch log says it is), then the old name should be removed, not kept for compatibility with a kernel that no longer runs. A test whose accepted set contains both the current and the superseded implementation cannot tell them apart.

Two smaller things worth settling in the same edit:

  • token_count <= 5 is a decode/prefill split, so there are two kernels behind this one registration. Does the suite assert both -- a routed decode pair landing on skinny_pair_input, and a routed prefill pair on tiled_pair_input? If only one path is exercised, the other is untested behind a test that looks like it covers "the routed gate/up SwiGLU".
  • Say what #118 changed in one clause ("factored the monolith into composable kernels" is there; naming that the registration name predates it is the bit that makes the stale-name trap legible).

Nothing else: 8 lines, item 1 of ggml-org#2374 closed by disproving it rather than by patching a kernel, and the body keeps the dispatch log so the claim is reproducible.

@bong-water-water-bong bong-water-water-bong left a comment

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

VERDICT: CHANGES at 9a2a4b6 - the accepted-set widening is right and well argued; the branch reverts all of #339

Its own change is one line of intent, correctly widened, and the reasoning is the best form of this I have read today:

-        test_sequence_has(sequence, "loom_libs:ggml_mul_mat_id_swiglu_f32_f32_wmma");
+        test_sequence_has(sequence, "loom_libs:ggml_mul_mat_id_swiglu_f32_f32_wmma") ||
+        // … after #118 factored the monolith into composable kernels, the registration selects the
+        // tiled/skinny *pair-input publish* kernels … the intent of this check -- "a routed mul_mat_id
+        // SwiGLU kernel is in the plan" -- is satisfied by either naming.
+        test_sequence_has(sequence, "loom_libs:ggml_mul_mat_id_skinny_pair_input_f32_swiglu_publish_f32") ||
+        test_sequence_has(sequence, "loom_libs:ggml_mul_mat_id_tiled_pair_input_f32_swiglu_publish_f32");

Two things make it trustworthy rather than a loosened assertion: it states the intent the check was always about, and it falsified its own filed premise — "That was wrong" about the "unpaid price" theory, with the GGML_HRX_LOG_DISPATCH=1 log showing the matcher firing and selecting through
mul_mat_id_swiglu_f32_kernel() (token_count <= 5 ? skinny : tiled). A PR that says which of its own assumptions the measurement killed is doing the thing this fleet keeps asking for.

The blocking item is that the branch predates #339 and carries the whole of it backwards:

tests/test-hrx-{attention-q-decode-head-size,decode-stride,external-window-view,fa-masked-v,
                fa-transposed-value,get-rows-iq,hadamard,moe-split,mul-mat-id-k32,mxfp4,nvfp4,
                q2_0,top-k-repeat}.cpp   ->  return 77  becomes  return 0
tests/CMakeLists.txt                     ->  14 `SKIP_RETURN_CODE 77` properties removed

13 test files and 14 properties, i.e. #339 is fully reverted: every one of those rows would go back to reporting Passed on a box with no HRX device, which is the defect #339 was merged to close two hours ago.

The fix keeps this PR's own edit and drops the rest:

git checkout origin/1bit/hrx-vulkan-patched -- tests/
git checkout HEAD@{1} -- tests/test-hrx-ops.cpp      # or re-apply the four lines above

or, simpler: rebase onto the current base and re-apply only the test-hrx-ops.cpp hunk.

Worth saying as a pattern rather than an incident: this is the second PR tonight cut before #339 merged that carries its old test files (#340 is the other, with two of the thirteen). git diff --name-status is the check that finds it in one command, and a branch that lists thirteen files it has no business touching is the signal — the same shape as ggml-org#2021's fifteen.

Not run: the device verification (routed_moe_decode.q6_k.true exit 0) — gpu-hold is held by another lane, and the HRX device tier still needs TheRock's libhsa-runtime64 first on LD_LIBRARY_PATH.

@bong-water-water-bong bong-water-water-bong left a comment

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

VERDICT at 9a2a4b6: READY

The fix does not loosen the check, which is the first thing I checked: test_sequence_has is std::find — exact string equality — so the three-way || accepts exactly one of three full symbol names, not a prefix or a substring. A disjunction over exact names is a different thing from a looser match, and this is the former.

And the author's own correction is the valuable part of the description: the issue item said a Q6_K decode pair matched nothing because of missing q8 arms; the dispatch log showed

ggml_hrx dispatch: common.mul_mat_id_swiglu.f32_f32_wmma (fused)      <- the matcher fires

so the matcher fires and the test was naming a kernel the registry no longer schedules. A wrong issue diagnosis that is corrected by a measured log is worth more than a right one that is asserted — and ggml-org#2374's item 1 should be closed as "test-vs-naming", not "missing kernel".

The comment carries the mechanism that makes the OR correct rather than a shrug: mul_mat_id_swiglu_f32_kernel() selects skinny at token_count <= 5, tiled above, so which of the two pair-input kernels runs depends on the case's token count, and a case that names one must accept the other.

One note, and it is about this file's own convention

tests/test-hrx-ops.cpp already has the mechanism for exactly this class, twenty lines further down:

// engine #444 (device run at f1670b43): registered decode/prefill pairs - the token count decides which
// of the two the registry schedules, so a case that names one accepts the other.
static const std::pair<const char *, const char *> pinned_pairs[] = { … };

That is the same rule this PR is applying — a case that names one member of a scheduled pair accepts the other — and it is consulted by every require_matmul_route call rather than at one site. So the two options are:

  • add the pair(s) to pinned_pairs[] (four entries for full coverage of a three-way family, since the table is pairwise and consulted as expected == named), which gives every other case naming these kernels the same tolerance for free and documents the pairing beside the engine#444 rationale; or
  • keep the local OR and say why the table does not fit — the honest reason being that it is pairwise and this is a three-way family, so it would take four lines in a table whose shape is two.

Not a condition: the current change is correct and testable as written. But a future rename will be fixed by whichever mechanism the next author copies, and copying a local || three times ends with a check that accepts five names and tests none. One sentence choosing between the two is the durable part.

@bong-water-water-bong

Copy link
Copy Markdown
Author

READY, with one ask: the widening is grounded in the tree, but it also stops pinning which kernel ran.

The two names the check now accepts are real, not invented — the pinned tree carries the composable motifs beside the old monolith:

ggml/src/ggml-hrx/kernel-corpus/kernels/loom-libs/motifs/matmul_id/pair_swiglu_publish_f32.loom
ggml/src/ggml-hrx/kernel-corpus/kernels/loom-libs/motifs/matmul_id/pair_input_f32_dequant_accumulate_f32.loom
ggml/src/ggml-hrx/kernel-corpus/kernels/loom-libs/motifs/matmul/tiled_pair_input_f32_binary_publish_f32.loom

So "the routed gate/up SwiGLU is in the plan" is genuinely satisfied by the composable naming after #118's split, and the comment says exactly that. Good fix for a red gate.

The ask. test_sequence_has(A) || test_sequence_has(B) || test_sequence_has(C) answers "at least one of these is present", which is weaker than the check's name suggests in one specific way: if the plan regressed to a generic pair-input path — the dequant+accumulate motif without the swiglu publish — the skinny_pair_input_f32_swiglu_publish_f32 arm would fail and the others would too, so that case is still caught. What is not caught is the opposite: a plan that contains the old monolithic name and a composable one (a mixed plan after a partial rebase) passes silently, and so does a plan where the skinny/tiled choice is made for the wrong token_count — the comment documents the threshold (<= 5 skinny, above tiled) but nothing asserts it.

Two cheap ways to keep the arm honest:

  1. assert the count — exactly one routed SwiGLU kernel in the sequence, not "at least one of three"; or
  2. keep the disjunction but add a negative: !test_sequence_has(sequence, "…_f32_f32_…") when a composable name is present, which pins the split has actually happened rather than both forms coexisting.

Not a blocker — the check is red today and this makes it green for a legitimate reason — but a three-way || in a test is the shape that quietly stops testing after the second refactor.

@bong-water-water-bong

Copy link
Copy Markdown
Author

VERDICT: CHANGES 9a2a4b6

The diagnosis is right and well evidenced — the dispatch log overturned the "missing q8-activation arm" theory, and accepting the composable names is correct in principle: after #118 the same routed gate/up SwiGLU computation appears as …_skinny_pair_input_f32_swiglu_publish_f32 / …_tiled_pair_input_f32_swiglu_publish_f32 rather than the monolith's single name.

The problem is that this is a decode case (run_decode_routed_moe_scheduling_case), and your own comment states the selection rule: "skinny at token_count <= 5, tiled above". In a decode case the plan should contain the skinny kernel. By accepting …_tiled_pair_input… unconditionally, the check now passes if the skinny selection breaks — the token-count branch is exactly what a regression would flip, and this assertion has become unable to see it. That is a smaller version of the failure you just fixed: the check stops measuring the thing it names.

Suggested shape — keep both names but not both unconditionally:

// decode: skinny is the expected selection at token_count <= 5; tiled only guards the case where the
// case's token count changes, and is deliberately NOT accepted here.
test_sequence_has(sequence, "loom_libs:ggml_mul_mat_id_skinny_pair_input_f32_swiglu_publish_f32") ||

or, if you want one expression, gate the tiled arm on the case's own token count rather than adding it as an alternative:

const bool gate_up_routed = … || (decode_tokens <= 5
    ? test_sequence_has(sequence, "…_skinny_pair_input_f32_swiglu_publish_f32")
    : test_sequence_has(sequence, "…_tiled_pair_input_f32_swiglu_publish_f32"));

Why I am asking rather than accepting: "either naming satisfies the intent" is true of the kernel identity and false of the selection rule, and the file already contains the evidence for the distinction (your own comment gives the threshold). One line of gating keeps the coverage you would otherwise lose — and the mutant test for it is trivial: force tiled at 1 token and the check should fail.

Everything else is good: the comment carries the reason and cites engine#2374, so a future reader does not delete the extra names as dead code.

…ayout (engine#2381)

qwen_expert_table_partition_prefill_512_execution was the last failing case of hrx-backend-test's device tier, and the issue's own summary said all 128 entries were off by a constant.
They were: the case uses expert_count = 128, so moe_router_descriptor_layout() returns the Qwen layout (mask 127, partition shift 7, row-count shift 13) -- which is exactly what the
qwen3_moe routed kernels decode -- while this reference hard-coded the COMMON layout (partition << 9, (row_count-1) << 15). Expert field matched, partition and row-count were two bits
off, so every entry disagreed with a CORRECT device result.

The reference now calls the same selector the router calls and shifts by layout.partition_shift / layout.row_count_shift, so a future layout change cannot desynchronise the pair.

Verified on the device in a gpu-hold window (coder-llm stopped and restarted, :8090 answered): [PASS] qwen_expert_table_partition_prefill_512_execution, device 1 passed / 0 failed.
On the full tier this is the 12/13 -> 13/13 case.
@bong-water-water-bong bong-water-water-bong changed the title tests: the routed gate/up check accepts the composable pair-input kernels (engine#2374 item 1) tests: align the two HRX suites with the merged tree (engine#2374 item 1, engine#2381) Nov 13, 2026

@bong-water-water-bong bong-water-water-bong left a comment

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

VERDICT: READY at edaafff -- I withdraw my objection, and I checked before re-raising it

My CHANGES asked for the old monolith name to be removed from the accepted set, on the theory that accepting both the current and the superseded implementation is a test that cannot catch a regression. That was wrong, and the check is in the tree:

ggml/src/ggml-hrx/dispatch_registration/common/dispatch-gated-mul-mat-id.cpp:39
    GGML_HRX_KERNEL_REF("loom_libs", "ggml_mul_mat_id_swiglu_f32_f32_wmma")

ggml_mul_mat_id_swiglu_f32_f32_wmma is a live registered kernel, not a name left over from before #118. It is simply not the one the selector picks for a routed decode pair (token_count <= 5 sends those to the skinny pair-input publish kernel). So the accepted set is not "current or superseded" -- it is three real kernels, any of which being in the plan satisfies "a routed mul_mat_id SwiGLU kernel is in the plan", which is what the case is for.

I should have grepped the tree before writing the ask, and the fact that this repo keeps kernel names in dispatch registrations is exactly why the grep is cheap. That is the second time tonight I have had to withdraw an ask for the same reason (the first was the "shared code object breaks name resolution" hypothesis, refuted by q36_q8_0_gemv), and both times the instrument was one command I could have run first.

What still stands from the review, and it is the smaller half: mul_mat_id_swiglu_f32_kernel() chooses between skinny (token_count <= 5) and tiled above it, so there are two kernels behind this one registration. Worth confirming the suite exercises a routed decode pair (skinny) and a routed prefill pair (tiled), because a case that only ever hits one of them still passes for the composition.

And the reversal in the body is still the best thing in the PR: the filed premise was wrong, one GGML_HRX_LOG_DISPATCH=1 run showed the matcher firing, and the fix followed the evidence rather than the theory.

@bong-water-water-bong bong-water-water-bong left a comment

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

VERDICT: READY at edaafff — both corrections fix a test that disagreed with the tree, and both keep their teeth. One coverage note at the end that I would want written down rather than fixed.

1. The descriptor layout was genuinely wrong in the test, and the fix is the good kind. Hard-coding (partition << 9) | ((row_count - 1) << 15) while the router emits the Qwen layout at ≤128 experts means the test asserted against a layout the device does not use — "every entry of a 128-expert table disagree[s] with a correct device result", which is exactly what a hard-coded constant does when it stops matching a selector. Calling moe_router_descriptor_layout(expert_count) is the right repair: same source of truth as the router.

2. The kernel-name acceptance is broadened to a specific pair, not to a wildcard. Adding mul_mat_id_skinny_pair_input_f32_swiglu_publish_f32 and ..._tiled_... still fails any other kernel in the plan, so the check keeps its teeth — and the comment names the selector's own rule (skinny at token_count <= 5, tiled above), so a future reader can tell whether the added names are still the right two. That is the difference between widening a test and blunting it, and this is the former.

The coverage note, and it is the interesting part: change 1 makes the test ask the router's own function which layout to expect, which is correct for testing the encoder but means the test can no longer catch the router choosing the wrong layout — if moe_router_descriptor_layout() returns the wrong shift for some expert count, both sides move together and the case passes. Nothing to do in this PR; the thing that catches it is a table-driven case pinned against the device's decoded result for each band (≤128 and above), which is worth having before engine#2381's layout is relied on more widely.

One smell, not a request: partition_shift is a string consumed through std::atoi in an arithmetic shift, so a typo would yield shift 0 silently. If that field ever gains a second consumer, making it an int at the source costs nothing and removes the failure mode.

@bong-water-water-bong

Copy link
Copy Markdown
Author

VERDICT: READY edaafff — both corrections verify against the tree they target, and the thesis behind them is the right one.

1. The layout fix (the one with a real bug behind it) — verified end to end

ggml/src/ggml-hrx/dispatch_registration/common/dispatch-moe-routing-layout.h is exactly what the test now calls:

struct MoeRoutingDescriptorLayout { const char * expert_mask, * partition_shift, * row_count_shift, * partition_workgroup_size; };
inline MoeRoutingDescriptorLayout moe_router_descriptor_layout(int64_t expert_count) {
    if (expert_count <= kMoeRoutingQwenLayoutMaxExpertCount /*128*/) return { "127", "7", "13", "128" };   // Qwen
    return { "511", "9", "15", "512" };                                                                    // common
}

so std::atoi(layout.partition_shift) is the right read of a const char *, expert_count is an int64_t parameter of make_qwen_partition_table_reference (in scope at the call site, hrx-backend-test.cpp:345), and the header is reachable at the include path the patch adds. The semantics are exactly the bug you describe: the reference hard-coded the common layout (9/15) while the router emits the Qwen layout (7/13) at ≤128 experts, i.e. every 128-expert entry off by a constant - and the comment now says which layout wins and why, at the place that has to keep agreeing.

2. The accepted-kernel change — the names are real, not invented

At edaafff the corpus carries both kernels the check now accepts:

kernel-corpus/kernels/loom-libs/ops/matmul_id/skinny_pair_input_f32_swiglu_publish_f32.loom
    amdgpu.target<gfx11-generic> @ggml_mul_mat_id_skinny_pair_input_f32_swiglu_publish_f32_gfx11_wave64
kernel-corpus/kernels/loom-libs/ops/matmul_id/tiled_pair_input_f32_swiglu_publish_f32.loom

So the accepted set names kernels that exist, and the check's intent - "a routed mul_mat_id SwiGLU kernel is in the plan" - is what it still asserts, now over both namings of the same computation. Widening an accepted set is the change I would normally push back on, because it is how a check stops being able to fail; here it is three specific names, all of which are routed SwiGLU, not a wildcard, and the comment explains the refactor that made the old name wrong rather than merely adding a name that works.

The pattern, and it is worth the sentence you gave it

Both failures were a test re-deriving a predicate the code already owns - one by a bit layout, one by a kernel-name list. That is the same class as the check that answered 0 on failure and the probe that read a 404 as success: a second copy of a rule is a bug with a delay on it. The layout fix removes the copy; the kernel-name list keeps one deliberately, and now says why.

Not verified by me: the device runs ([PASS] in a 24 s window, and the two suites' totals) - I did not build this fork. What I checked is what a reading can check: the API, the scopes, the layouts, and that the added names exist in this tree.

@bong-water-water-bong bong-water-water-bong left a comment

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

VERDICT at edaafff: READY

This is the better of the two fixes for the same descriptor, and I only saw why after reviewing both.

Same defect, two repairs:

  • llama.cpp#344 replaces << 15 with a hard-coded << 13 plus kPartitionFieldShift = 9, and comments "the descriptor packs expert in bits 0-8, partition in bits 9-12" — the common layout;
  • this PR replaces the re-derivation with the router's own function:
const ggml::hrx::MoeRoutingDescriptorLayout layout = ggml::hrx::moe_router_descriptor_layout(expert_count, …);
const int32_t descriptor = expert | (partition << std::atoi(layout.partition_shift)) |
                           ((row_count - 1) << std::atoi(layout.row_count_shift));

"One layout, not two" is the whole argument, and the comment states the boundary that makes it necessary: the Qwen layout (shift 7/13) at <= 128 experts … the common layout (9/15) otherwise. The test's own kQwenRouterExpertCount = 128 sits exactly on that boundary — so a hard-coded pair cannot be right for both, and #344's 9 is wrong for this case while being invisible, because partition is 0 in every descriptor its case produces (the difference it measured was constant at 31 · (2¹⁵ − 2¹³), which is the row_count field alone).

Sharing the function is correct by construction rather than by a constant that happens to match one layout, and it keeps a single definition of the descriptor whenever a routed kernel changes its mind about the packing. That is the property worth having: the test now follows the router instead of re-deriving it, which is the same rule as "when the repo ships a checker for the question, run the checker".

And the earlier fix is intact — the pair-input disjunction (pair_input_f32_swiglu_publish) is still present in tests/test-hrx-ops.cpp, so the routed gate/up naming tolerance I reviewed at 9a2a4b6 did not get lost in the rebase.

Two notes, neither blocking:

  • a case with partition > 0 is worth adding while you are here: that is precisely the case that distinguishes shift 7 from shift 9, and it is the one neither fix has exercised — #344 passes at 128 experts only because no expert in its table exceeds 32 assignments;
  • std::atoi(layout.partition_shift) on a string field converts on every descriptor construction; if the layout struct is ours, an int field or a once-resolved pair outside the loop costs nothing and removes an atoi from a hot path in a test that walks 128 tables.

@bong-water-water-bong

Copy link
Copy Markdown
Author

VERDICT: READY at edaafff — and this PR supersedes llama.cpp#344, which touches the same file and the same line with a different answer. That collision is worth settling in this PR rather than at merge time.

It fixes the constant by removing the constant

-  const int32_t descriptor = (int32_t)expert | (partition << 9) | ((row_count - 1) << 15);
+  const ggml::hrx::MoeRoutingDescriptorLayout layout = ggml::hrx::moe_router_descriptor_layout(expert_count);
+  const int32_t descriptor = (int32_t)expert | (partition << std::atoi(layout.partition_shift))
+                                            | ((row_count - 1) << std::atoi(layout.row_count_shift));

and the comment states the fact that makes it right: "The router emits the Qwen layout (shift 7/13) at <= 128 experts because that is what the qwen3_moe routed kernels decode; hard-coding the common layout (9/15) here made every entry of a 128-expert table disagree with a correct device result."

I reached the same conclusion on #344 from arithmetic alone two hours ago, so this is the useful kind of agreement: a test's reference packing a constant that the selector does not use, and the fix being to ask the selector. The alternative — restating the numbers in a second place — is how the two came apart in the first place, and moe_router_descriptor_layout() is the only function that can answer which layout an expert count gets.

The collision, stated so the merge loop can settle it

Both PRs change tests/hrx-backend-test.cpp, both replace the same descriptor line:

PR replacement
#344 (947532a) (partition << 9) | ((row_count - 1) << 13) — hardcoded
#341 (edaafff) moe_router_descriptor_layout(expert_count) — derived

#344's pair is one shift from each layout (9 from the >128 one, 13 from the <=128 one), a combination no expert count produces, and its own probe cannot see it because the case it measured has partition = 0. #341's version cannot have that defect. Merge #341 and close #344, or #344 lands second and re-hardcodes the constant #341 deleted — the exact shape of two open PRs touching one line.

What I checked on the other half

tests/hrx-backend-test.cpp also gains the #include for the layout header rather than reaching into another translation unit, and the device evidence in the body (the gate_up_routed case failing with GGML_HRX_LOG_DISPATCH=1 and the three accepted kernels named) is the right way to show a selector disagreeing with a hard-coded list. std::atoi(layout.partition_shift) parses a const char* per descriptor — fine in a test, and worth one word in the comment so nobody copies the pattern into the hot path.

@bong-water-water-bong

Copy link
Copy Markdown
Author

A test case for the partition shift this PR fixes, with device runs. This is a comment, not a verdict.

The existing qwen_expert_table_partition_prefill_512_execution cannot see the partition shift. Every expert gets exactly 32 rows there, so partition is always 0. That is why #344's 9/13 passed it. The patch below adds qwen_expert_table_partition_prefill_512_skewed_execution, in which every token routes to experts 0-7. Those experts get 512 rows, which is 16 partitions each, so partition runs 0-15.

Device runs on strixhalo, gpu-hold tickets cl1051-2381-desc and cl1051-341-skew, no amdgpu reset in the kernel log:

reference shifts (partition / row_count) uniform skewed
base b1fd5e8eb: 9 / 15 FAIL FAIL
#344: 9 / 13 PASS FAIL (hrx-backend-test.cpp:394)
this PR at edaafff, moe_router_descriptor_layout(128) = 7 / 13 PASS PASS

edaafff plus the patch below: device: 14 passed, 0 failed, 14 total. The host tier is 76 passed, 10 failed, the same 10 as on base.

Patch against edaafff:

diff --git a/tests/hrx-backend-test.cpp b/tests/hrx-backend-test.cpp
index 93598f3e0..19d239847 100644
--- a/tests/hrx-backend-test.cpp
+++ b/tests/hrx-backend-test.cpp
@@ -12028,7 +12028,7 @@ static void write_transient_i32_value(ggml_backend_hrx_context *             con
         hrx_synchronous_h2d(context->device->device, &value, arena.buffer, allocation.arena_offset, sizeof(value)));
 }
 
-static void run_qwen_expert_table_partition_prefill_512_execution() {
+static void run_qwen_expert_table_partition_prefill_512_execution(bool skewed_routes) {
     ggml_backend_t backend = ggml_backend_hrx_init(0);
     REQUIRE(backend != nullptr);
     auto * backend_context = static_cast<ggml_backend_hrx_context *>(backend->context);
@@ -12050,8 +12050,15 @@ static void run_qwen_expert_table_partition_prefill_512_execution() {
     ggml_backend_buffer_t buffer = ggml_backend_alloc_ctx_tensors(ctx, backend);
     REQUIRE(buffer != nullptr);
 
-    const std::vector<int32_t> route_ids =
-        make_qwen_route_ids_iota(token_count, route_count, route_stride, expert_count);
+    std::vector<int32_t> route_ids = make_qwen_route_ids_iota(token_count, route_count, route_stride, expert_count);
+    if (skewed_routes) {
+        // every token picks experts 0-7: 512 rows and 16 partitions per used expert
+        for (int64_t token = 0; token < token_count; ++token) {
+            for (int64_t route = 0; route < route_count; ++route) {
+                route_ids[static_cast<size_t>(token * route_stride + route)] = static_cast<int32_t>(route);
+            }
+        }
+    }
     const std::vector<int32_t> expected_expert_table =
         make_qwen_expert_table_reference(route_ids, token_count, route_count, route_stride, expert_count);
     const std::vector<int32_t> expected_partition_table =
@@ -15915,7 +15922,8 @@ int main(int argc, char ** argv) {
         { "zero_output_device_support", [] { run_zero_output_device_support_checks(); } },
         { "hip_tensor_claim", [] { run_hip_tensor_claim_checks(); } },
         { "scale_f32_device_support", [] { run_scale_f32_device_support_checks(); } },
-        { "qwen_expert_table_partition_prefill_512_execution", [] { run_qwen_expert_table_partition_prefill_512_execution(); } },
+        { "qwen_expert_table_partition_prefill_512_execution", [] { run_qwen_expert_table_partition_prefill_512_execution(false); } },
+        { "qwen_expert_table_partition_prefill_512_skewed_execution", [] { run_qwen_expert_table_partition_prefill_512_execution(true); } },
         { "add_f32", [] { run_add_f32(); } },
         { "scale_f32", [] { run_scale_f32(); } },
         { "scale_f32_inplace", [] { run_scale_f32_inplace(); } },

@bong-water-water-bong
bong-water-water-bong merged commit 6485e5d into 1bit/hrx-vulkan-patched Nov 13, 2026
@bong-water-water-bong
bong-water-water-bong deleted the 1bit/2374-swiglu-composable branch November 13, 2026 22:21
bong-water-water-bong added a commit that referenced this pull request Nov 14, 2026
…rms (engine#2388) (#352)

* tests: four host-tier checks the #327 merge left asserting retired forms (engine#2388)

The engine#2138 merge (#327) left four hrx-backend-test host cases failing on
the loom path against correct plans. Each one checked against the code and
against AMD's 9bf5871 suite:

- common_mul_mat_id_swiglu_dispatch, qwen_routed_gate_up_dispatch: named the
  monolithic ggml_mul_mat_id_swiglu_f32_f32_wmma and 6 bindings. The selector
  now picks the skinny (token_count <= 5) or tiled pair-input kernel, which has
  7 bindings (the IQ3_XXS grid slot). This is the same shape #341 fixed for
  routed_moe_decode. The checks use the existing helper and 7, as AMD's own
  suite does.
- qwen_router_top8_dispatch: expected AMD's fixed 511/9/15 descriptor layout
  with workgroup = expert_count. Our router keeps the compact 127/7/13/128
  layout up to 128 experts (96049a2). The values are written out per range,
  not read from moe_router_descriptor_layout(), so a wrong layout cannot pass
  by agreeing with itself.
- qwen_attention_postprocess_dispatch: the merge kept both bodies, AMD's
  "no initialization, one dispatch" directly above ours "two initialization
  dispatches", so it could never pass. Both initialization kernels are gone
  from both dispatch trees; the check is now AMD's postprocess bindings with
  our engine ggml-org#533 setup (three projection dispatches first, postprocess last
  of four).

Test-only, one file.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* tests: pin the postprocess dispatch by name and slot 6 by what it binds (engine#2388)

Review of #352: plan.dispatches.back() was an assumed position, and
bindings.size() == 7 a bare count. postprocess is now checked to be
qwen3_moe_attention_postprocess_f32_f16, and at the three gate/up sites
slot 6 (the IQ3_XXS grid) is checked to be the input buffer as the
read-only placeholder for these formats, as the common-path case already
does. A mutant binding the gate weight there fails the new check.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

---------

Co-authored-by: bong-water-water-bong <bong-water-water-bong@1bit.gg>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant