Skip to content

tests(hrx): assert the add-on's packed-layout consequence instead of the loom name (engine #2392) - #360

Closed
bong-water-water-bong wants to merge 3 commits into
1bit/hrx-vulkan-patchedfrom
2392-addon-aware-host-tier
Closed

bong-water-water-bong wants to merge 3 commits into
1bit/hrx-vulkan-patchedfrom
2392-addon-aware-host-tier

Conversation

@bong-water-water-bong

Copy link
Copy Markdown

With the HIP add-on installed, hrx-backend-test's host tier failed 22–27 cases while both paths chose the same weight format (engine ggml-org#2392). This fixes the largest class properly and leaves the rest visible.

The ten packed_layout_follows_weight.* cases: a real difference, not an over-specified name

The add-on registers one kernel per weight format and never takes the q4k/q6k packed-tail path, so it never emits the Q4KRow64/Q6KRow64 layout binding nor the ggml.mul_mat.weight_format compile parameter that records the choice (dispatch-mul-mat.cpp:1217-1235 is where the loom path does both). The old assertion required that parameter, so found == 1 failed — correctly observing a difference.

The assertion now has two arms:

loom producer  -> REQUIRE(found == 1); weight_format == stored (+40 when file-mapped)   // unchanged
add-on kernel  -> REQUIRE(no binding carries a row-64 layout)                            // the consequence, in the open

The second arm states what actually happens instead of skipping the case, and it fails the day the add-on learns to pack — so the test cannot silently stop checking the packed layout.

Why the add-on cannot be identified by format

I assumed from hrx_kernel_ids.h that add-on kernels were named hrx_dp_mm_<format>_…. Printing the serving kernels showed otherwise:

serving kernel: hip:k_a074247a0b6a
serving kernel: hip:k_3757f9bc3e52

They are registered under the hip: namespace with opaque hash ids — the k_<hash> values the HRX_ID_* macros expand to. The format↔id tie belongs to the add-on's own ABI check (372 kernels match their code objects), not to the plan. kernel_is_addon_implementation() therefore tests the namespace, with a shortcut: comment to tighten it to a format check if readable names ever appear.

Measured

arm before after
HIP add-on 27 failed / 59 passed 17 failed / 69 passed
loom (no add-on) 76 / 10 76 / 10, same ten names

The loom arm was measured before and after on the same tree with the same config (stash → rebuild → run → restore), because the whole point of a two-arm assertion is that the configuration that already worked keeps its verdict. It does.

What is deliberately not here

17 failures remain, in four groups, none of them relaxed by this PR:

  • 2 tied_weight_few_tokens_keeps_kquant.* — the sibling of the ten, in schedule_tied_weight_few_tokens_keeps_kquant; same treatment available in a follow-up;
  • 4 dispatches.size() — the add-on fuses differently; needs per-case judgement about whether the fused plan is equivalent;
  • 4 name lookups (producer/fused/kernel_name) — the accepted-set treatment, same as above;
  • 4 alternate / gate_up_alternate != nullptr — add-on defects: the producer does not publish the alternate the consumer needs. These must be reported to the add-on's owner, not converted into passing tests.

Note the arm cannot run on gfx1201 today: the add-on's fa_prompt_* kernels use the gfx11 wmma intrinsic form and fail to compile for gfx1201, so this arm is built with -DGGML_HRX_HIP_TARGETS=gfx1151 (host tier needs plans only). That portability gap is tracked on ggml-org#2392.

Build: BUILD_RC=0, no errors — checked separately from the run, after I spent three runs measuring a stale binary because a build piped through grep hides its status.

bcloud and others added 3 commits November 13, 2026 16:42
…nels (engine#2374 item 1)

routed_moe_decode.q6_k.true failed with 'gate_up_routed', which I first read as our dispatch missing AMD's q8-activation arms after the engine#2138 decision. It is not:
with GGML_HRX_LOG_DISPATCH=1 the plan shows the matcher firing --

    ggml_hrx dispatch: common.mul_mat_id_swiglu.f32_f32_wmma (fused)

-- and that registration selects through mul_mat_id_swiglu_f32_kernel(), which returns the SKINNY pair-input kernel at token_count <= 5 and tiled above. Those are
the composable kernels upstream #118 produced when it factored the monolith. So the code launches the right kernel and the test's accepted set predates the refactor:
a Q6_K decode pair correctly lands on ggml_mul_mat_id_skinny_pair_input_f32_swiglu_publish_f32, which the case did not list.

The check's intent -- 'a routed mul_mat_id SwiGLU kernel is in the plan' -- is satisfied by either naming, and the case already accepts three alternatives, so the
two composable names join that set with the reason in a comment.

Verified on the device: the case no longer aborts (exit 0, no requirement failure).
…ayout (engine#2381)

qwen_expert_table_partition_prefill_512_execution was the last failing case of hrx-backend-test's device tier, and the issue's own summary said all 128 entries were off by a constant.
They were: the case uses expert_count = 128, so moe_router_descriptor_layout() returns the Qwen layout (mask 127, partition shift 7, row-count shift 13) -- which is exactly what the
qwen3_moe routed kernels decode -- while this reference hard-coded the COMMON layout (partition << 9, (row_count-1) << 15). Expert field matched, partition and row-count were two bits
off, so every entry disagreed with a CORRECT device result.

The reference now calls the same selector the router calls and shifts by layout.partition_shift / layout.row_count_shift, so a future layout change cannot desynchronise the pair.

Verified on the device in a gpu-hold window (coder-llm stopped and restarted, :8090 answered): [PASS] qwen_expert_table_partition_prefill_512_execution, device 1 passed / 0 failed.
On the full tier this is the 12/13 -> 13/13 case.
…the loom name (engine ggml-org#2392)

With the HIP add-on installed, the host tier failed 22-27 cases while both paths
chose the same weight format. For the ten packed_layout_follows_weight.* cases the
cause is concrete: the add-on registers one kernel per format and never takes the
q4k/q6k packed-tail path, so it never emits the Q4KRow64/Q6KRow64 layout binding
nor the ggml.mul_mat.weight_format compile parameter that records it.

The assertion now has two arms:
  loom producer  -> unchanged: weight_format == stored (+40 when file-mapped),
                    i.e. the packed layout still has to follow the weight;
  add-on kernel  -> REQUIRE that no binding carries a row-64 layout, stating the
                    consequence in the open rather than skipping the case.

The add-on's kernels are registered as hip:k_<hash> (opaque ids from HRX_ID_*
macros), so a format cannot be read off their names; kernel_is_addon_implementation()
tests the namespace, and the format<->id tie stays with the add-on's own ABI check.
shortcut: namespace test; tighten to a format check if readable names appear.

Measured: add-on arm 27 -> 17 host failures (86 total). Loom arm unchanged at
76 passed / 10 failed with the same ten names before and after this change on the
same tree, so the configuration that already worked keeps its verdict.

17 failures remain and are not addressed here: 2 tied_weight kquant (same pattern),
4 dispatches.size() (fusion shape), 4 name lookups, and 4 missing alternates which
are add-on defects rather than test over-specification.
@bong-water-water-bong

Copy link
Copy Markdown
Author

Superseded by the lead's decision on engine#2392 (truthful labels) and by #359 which implements it — closing rather than competing with the owner's PR. I had gone the other way (two-arm assertions: loom arm unchanged, add-on arm asserts the observable consequence), and measured why the ten packed_layout cases fail: the add-on registers one kernel per format and never takes the q4k/q6k packed-tail path, so it emits neither the Q4KRow64/Q6KRow64 layout binding nor the weight_format parameter. Full per-case mechanism data is on ggml-org#2392 and I'm posting it on #359 where the label table needs it.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant