Repository navigation
tests(hrx): assert the add-on's packed-layout consequence instead of the loom name (engine #2392) - #360
Closed
bong-water-water-bong wants to merge 3 commits into
Closed
tests(hrx): assert the add-on's packed-layout consequence instead of the loom name (engine #2392)#360bong-water-water-bong wants to merge 3 commits into
bong-water-water-bong wants to merge 3 commits into
Conversation
…nels (engine#2374 item 1)
routed_moe_decode.q6_k.true failed with 'gate_up_routed', which I first read as our dispatch missing AMD's q8-activation arms after the engine#2138 decision. It is not:
with GGML_HRX_LOG_DISPATCH=1 the plan shows the matcher firing --
ggml_hrx dispatch: common.mul_mat_id_swiglu.f32_f32_wmma (fused)
-- and that registration selects through mul_mat_id_swiglu_f32_kernel(), which returns the SKINNY pair-input kernel at token_count <= 5 and tiled above. Those are
the composable kernels upstream #118 produced when it factored the monolith. So the code launches the right kernel and the test's accepted set predates the refactor:
a Q6_K decode pair correctly lands on ggml_mul_mat_id_skinny_pair_input_f32_swiglu_publish_f32, which the case did not list.
The check's intent -- 'a routed mul_mat_id SwiGLU kernel is in the plan' -- is satisfied by either naming, and the case already accepts three alternatives, so the
two composable names join that set with the reason in a comment.
Verified on the device: the case no longer aborts (exit 0, no requirement failure).
…ayout (engine#2381) qwen_expert_table_partition_prefill_512_execution was the last failing case of hrx-backend-test's device tier, and the issue's own summary said all 128 entries were off by a constant. They were: the case uses expert_count = 128, so moe_router_descriptor_layout() returns the Qwen layout (mask 127, partition shift 7, row-count shift 13) -- which is exactly what the qwen3_moe routed kernels decode -- while this reference hard-coded the COMMON layout (partition << 9, (row_count-1) << 15). Expert field matched, partition and row-count were two bits off, so every entry disagreed with a CORRECT device result. The reference now calls the same selector the router calls and shifts by layout.partition_shift / layout.row_count_shift, so a future layout change cannot desynchronise the pair. Verified on the device in a gpu-hold window (coder-llm stopped and restarted, :8090 answered): [PASS] qwen_expert_table_partition_prefill_512_execution, device 1 passed / 0 failed. On the full tier this is the 12/13 -> 13/13 case.
…the loom name (engine ggml-org#2392) With the HIP add-on installed, the host tier failed 22-27 cases while both paths chose the same weight format. For the ten packed_layout_follows_weight.* cases the cause is concrete: the add-on registers one kernel per format and never takes the q4k/q6k packed-tail path, so it never emits the Q4KRow64/Q6KRow64 layout binding nor the ggml.mul_mat.weight_format compile parameter that records it. The assertion now has two arms: loom producer -> unchanged: weight_format == stored (+40 when file-mapped), i.e. the packed layout still has to follow the weight; add-on kernel -> REQUIRE that no binding carries a row-64 layout, stating the consequence in the open rather than skipping the case. The add-on's kernels are registered as hip:k_<hash> (opaque ids from HRX_ID_* macros), so a format cannot be read off their names; kernel_is_addon_implementation() tests the namespace, and the format<->id tie stays with the add-on's own ABI check. shortcut: namespace test; tighten to a format check if readable names appear. Measured: add-on arm 27 -> 17 host failures (86 total). Loom arm unchanged at 76 passed / 10 failed with the same ten names before and after this change on the same tree, so the configuration that already worked keeps its verdict. 17 failures remain and are not addressed here: 2 tied_weight kquant (same pattern), 4 dispatches.size() (fusion shape), 4 name lookups, and 4 missing alternates which are add-on defects rather than test over-specification.
Author
|
Superseded by the lead's decision on engine#2392 (truthful labels) and by #359 which implements it — closing rather than competing with the owner's PR. I had gone the other way (two-arm assertions: loom arm unchanged, add-on arm asserts the observable consequence), and measured why the ten packed_layout cases fail: the add-on registers one kernel per format and never takes the q4k/q6k packed-tail path, so it emits neither the Q4KRow64/Q6KRow64 layout binding nor the weight_format parameter. Full per-case mechanism data is on ggml-org#2392 and I'm posting it on #359 where the label table needs it. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
With the HIP add-on installed,
hrx-backend-test's host tier failed 22–27 cases while both paths chose the same weight format (engine ggml-org#2392). This fixes the largest class properly and leaves the rest visible.The ten
packed_layout_follows_weight.*cases: a real difference, not an over-specified nameThe add-on registers one kernel per weight format and never takes the q4k/q6k packed-tail path, so it never emits the
Q4KRow64/Q6KRow64layout binding nor theggml.mul_mat.weight_formatcompile parameter that records the choice (dispatch-mul-mat.cpp:1217-1235is where the loom path does both). The old assertion required that parameter, sofound == 1failed — correctly observing a difference.The assertion now has two arms:
The second arm states what actually happens instead of skipping the case, and it fails the day the add-on learns to pack — so the test cannot silently stop checking the packed layout.
Why the add-on cannot be identified by format
I assumed from
hrx_kernel_ids.hthat add-on kernels were namedhrx_dp_mm_<format>_…. Printing the serving kernels showed otherwise:They are registered under the
hip:namespace with opaque hash ids — thek_<hash>values theHRX_ID_*macros expand to. The format↔id tie belongs to the add-on's own ABI check (372 kernels match their code objects), not to the plan.kernel_is_addon_implementation()therefore tests the namespace, with ashortcut:comment to tighten it to a format check if readable names ever appear.Measured
The loom arm was measured before and after on the same tree with the same config (stash → rebuild → run → restore), because the whole point of a two-arm assertion is that the configuration that already worked keeps its verdict. It does.
What is deliberately not here
17 failures remain, in four groups, none of them relaxed by this PR:
tied_weight_few_tokens_keeps_kquant.*— the sibling of the ten, inschedule_tied_weight_few_tokens_keeps_kquant; same treatment available in a follow-up;dispatches.size()— the add-on fuses differently; needs per-case judgement about whether the fused plan is equivalent;producer/fused/kernel_name) — the accepted-set treatment, same as above;alternate/gate_up_alternate != nullptr— add-on defects: the producer does not publish the alternate the consumer needs. These must be reported to the add-on's owner, not converted into passing tests.Note the arm cannot run on gfx1201 today: the add-on's
fa_prompt_*kernels use the gfx11 wmma intrinsic form and fail to compile for gfx1201, so this arm is built with-DGGML_HRX_HIP_TARGETS=gfx1151(host tier needs plans only). That portability gap is tracked on ggml-org#2392.Build:
BUILD_RC=0, no errors — checked separately from the run, after I spent three runs measuring a stale binary because a build piped throughgrephides its status.