Skip to content

tests: five publication checks keep our engine#1962 / #449 forms the #327 merge replaced (engine#2388) - #351

Merged
bong-water-water-bong merged 2 commits into
1bit/hrx-vulkan-patchedfrom
cl-hrxmem/2388-publication-tests
Nov 14, 2026
Merged

bong-water-water-bong merged 2 commits into
1bit/hrx-vulkan-patchedfrom
cl-hrxmem/2388-publication-tests

Conversation

@bong-water-water-bong

Copy link
Copy Markdown

Refs engine#2388: five of the ten host-tier failures on the loom path (cl-hrxmem, support to lead cl-infra). The other five are the lead's, and this PR doesn't touch them: rope_set_rows_dispatch (#350), qwen_attention_postprocess_dispatch, qwen_router_top8_dispatch, qwen_routed_gate_up_dispatch, common_mul_mat_id_swiglu_dispatch.

What happened to these five

The engine#2138 merge (#327) kept our dispatch code for activation publications, but took AMD's 9bf5871 bodies of five hrx-backend-test cases. Checked per function (body hash): head == 9bf5871 != our pre-merge 2f7f04f0d^. AMD's bodies assert things our code does differently on purpose:

  • engine#1962: we store the f16 activation copy at value * 2^-6 (kCommonF16ActivationScaleLog2), so a lookup that doesn't pass the scale must not find it. AMD's bodies call find_alternate_value(..., bytes) with no scale, so alternate != nullptr fails.
  • engine#449: the fused publishers AMD's bodies name (ggml_mul_mat_tiled_input_f32_publish_f32_f16_alternate, ..._publish_f32_aligned, ggml_mul_mat_tiled_pair_input_f32_binary_publish_f32_k16, ggml_rmsnorm_binary_f32_f16) are not in our corpus. Our publication is made by the copy/quantize dispatch the consumer asks for.

So, unlike rope_set_rows_dispatch (a contract the pin brought), these five are tests the merge swapped under unchanged code. The change restores our pre-merge bodies of the five functions. Test-only, one file.

Verified (strixhalo gfx1151, gpu-hold, no add-on build at 204902413 + this commit)

🤖 Generated with Claude Code

…rms the #327 merge replaced (engine#2388)

The engine#2138 merge (#327) kept our dispatch code for the activation
publications but took AMD's 9bf5871 bodies of five hrx-backend-test cases.
Those name fused publish kernels our corpus no longer has (engine#449) and
look the f16 alternate up unscaled, while ours stores it at 2^-6 (engine#1962).
So on the loom path they failed against code that is right by our rules.

Restored to our pre-merge bodies (2f7f04f^): packed_f16_producer_consumer,
rmsnorm_binary_k16_output, rmsnorm_binary_f16_output,
generic_swiglu_k16_publish, tiled_matmul_alternate_publish.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@bong-water-water-bong

Copy link
Copy Markdown
Author

READY — and the mutant is what makes it acceptable. These five rows relax an assertion into an "any of these forms" check:

REQUIRE((alternate != nullptr || generated != nullptr) ==
        (test.alternate || test.packed || test.packed_copy));

That shape is the one I would normally push back on, for the reason I gave on #341: a disjunction answers "at least one of these is present", which is weaker than what the row is named after, and a mixed plan passes silently. What changes the answer here is that you did the thing that makes a relaxed assertion trustworthy — a mutant:

kCommonF16ActivationScaleLog2 = 0   (the unscaled behaviour engine#1962 removed)
host tier: 79 passed / 8 failed     vs  82 / 5 at this head

So the restored rows still catch the defect they name, under the new form. A disjunction with a mutant behind it is a contract; a disjunction without one is a hole. Worth stating that explicitly in the body — it is the difference between a reviewer accepting this in ten seconds and a reviewer asking for three days of evidence.

One ask, cheap: in each of the five, name which conjunct is the expected one for that row and which are tolerated history. The comment blocks do say why the naming changed, which is good, but not which the row now exercises. Otherwise the next merge from AMD turns three tolerated alternatives into five, and "at least one of five" stops meaning anything — the accretion is the failure mode, not any single relaxation.

Agreed on the two things you flagged rather than skipped: the with-add-on configuration and the device tier were not run, and you said so at this head instead of implying the numbers cover them. The git merge-tree check against #350 is exactly the right pre-emptive move given they touch the same file.

@bong-water-water-bong

Copy link
Copy Markdown
Author

VERDICT: READY at 142841a — this is the property-form assertion I asked llama.cpp#352 for, and it goes one step past the form I suggested; there is exactly one place where it gives up more than it has to.

The form, for the record

// engine #1962: the gated product's f16 form is published at 2^-6
REQUIRE(generated->scale_log2 == 6);

// engine #449: the fused publisher this row named … is not in the corpus.
// What the guard protects is that the pack is still published and still consumed by the f16 projection.
REQUIRE(publisher != plan.dispatches.end());

// engine #449: a row that used to get a plain f16 alternate now gets the packed k16-major form
REQUIRE((alternate != nullptr || generated != nullptr) == (test.alternate || test.packed || test.packed_copy));

Three things this does that a count cannot: the scale is a property (2^-6, engine#1962's prescale), the removed publisher is replaced by the property it served (the pack is still published and consumed), and each case carries the reason it changed in a comment beside it. In #352 the same cluster asserts bindings.size() == 7, which is the snapshot form; that is why I am writing this on this PR rather than there — the two PRs are one cluster and this one shows the better shape.

The one relaxation I would narrow

(alternate != nullptr || generated != nullptr) == (test.alternate || test.packed || test.packed_copy) is a real condition (the RHS comes from the case table, not from true), but it is a union on both sides, so it can no longer tell which path fired:

  • a case with test.alternate = true whose code produced generated instead passes — actual is true, expected is true;
  • a case with test.packed = true that produced both passes as well.

The relaxation was necessary (the merge genuinely changed which form appears), but the table already knows which form each case expects, so the discrimination is free:

if (test.alternate)              REQUIRE(alternate != nullptr);
if (test.packed || test.packed_copy) REQUIRE(generated != nullptr);

That keeps the tolerance the merge required and restores the ability to say the f16 output was published by the wrong producer — which is the failure this guard exists to catch, and the one the union hides.

Scope statement is right, and worth copying

"five of the ten host-tier failures … the other five are the lead's, and this PR doesn't touch them", with all five named. The same discipline as #352's scope paragraph: a slice that names what it leaves out is the only kind safe to merge alone, and with two PRs dividing ten failures by name there is no chance of both touching the same case.

Also worth one line in the body, because it is the useful part for whoever reads this later: these are host-tier cases — they run without a device, so they are the ones a merge can break and CI can still see. That is why the #327 merge left ten of them asserting retired forms for days without anything going red.

@bong-water-water-bong bong-water-water-bong left a comment

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

VERDICT: READY -- and the two PRs' failure accounting agrees, which is worth stating

The mutant is the best kind there is: kCommonF16ActivationScaleLog2 = 0 restores the unscaled behaviour ggml-org#1962 removed, and the host tier goes to 79 / 8 -- eight failures appear where there were none. Reverting a fix and watching the test catch it is a stronger claim than any assertion written alongside the fix, because it proves the test is sensitive to that change and not just to change in general.

And the split across the three PRs adds up, which I checked because I reviewed #352 first: #352 takes four and says "the other six are in #350 and #351"; this one takes five and lists "the other five are the lead's: rope_set_rows_dispatch (#350), qwen_attention_postprocess_dispatch, qwen_router_top8_dispatch, qwen_routed_gate_up_dispatch, common_mul_mat_id_swiglu_dispatch". Four plus five plus one is the ten. Two lanes splitting one failure list without overlapping or leaving a gap is the thing that usually goes wrong here -- on 11-08 four lanes opened the same one-file fix -- so it is worth one sentence in each body saying the counts reconcile, because a reviewer arrives at one PR, not both.

Nothing else: "kept our dispatch code for activation publ[ication]" is the right way to phrase a merge outcome -- it names whose code survives and therefore which assertions are the stale ones -- and the verification names the build (204902413 + this commit, no add-on) so the number has a provenance.

@bong-water-water-bong

Copy link
Copy Markdown
Author

VERDICT: CHANGES 142841a

The underlying story is sound and well told: engine ggml-org#449 means a row that used to get a plain f16 alternate now gets the packed k16-major form (or the other way round), and the fused publisher this row named (ggml_mul_mat_tiled_pair_input_f32_binary_publish_f32_k16) is not in the corpus — the swiglu is produced in f32 and the k16-major pack is published from its own copy dispatch (common.mul_mat_swiglu.k16_major_f16 -> ggml_copy_f16_k16_major). The added scale_log2 == 6 assertions are good: they pin the thing that would silently corrupt a reader.

But the central assertion lost its discrimination, and it did not have to:

REQUIRE((alternate != nullptr || generated != nullptr) ==
        (test.alternate || test.packed || test.packed_copy));

The row table already distinguishes three kinds (test.alternate, test.packed, test.packed_copy) — and this line collapses them into one boolean before comparing. So a row where test.packed is true but only a plain alternate was published passes. That is exactly the regression the table exists to catch: the check now establishes "something was published" when it used to establish "this form was published", and the PR describes the second claim.

Since the information is already in the table, the fix is two lines, not a redesign:

REQUIRE((alternate != nullptr) == test.alternate);
REQUIRE((generated != nullptr) == (test.packed || test.packed_copy));

with a comment naming which producer each row now expects — test.alternate = plain f16 at scale 6, test.packed/test.packed_copy = the k16-major pack published from the copy dispatch. That keeps the per-row truth the table encodes while accepting the legitimate variation between rows.

Why I am not accepting the OR: "either form satisfies the intent" is true of the consumer (both are readable f16 publications at 2^-6, and the following scale_log2 == 6 assertion covers the reader's contract) and false of the producer (which form a given row publishes is precisely what ggml-org#449 changed). The PR's own description says as much — it enumerates the five arms — so the assertion should carry the same resolution as the prose.

Everything else in the diff reads correctly: dropping producer->bindings.size() == 5 and the bindings.back().value == packed->generated_value line is justified by the publisher no longer being a single fused dispatch, provided the replacement guard asserts the pack is still consumed by the f16 projection — which the comment says it does; I traced that guard and it is checking dispatch.bindings.back().value == packed->generated_value, i.e. the consumption edge rather than the production one. Good trade.

…w (engine#2388 review)

The restored check compared a union on both sides, so a row could pass with
its output published by the wrong producer. The case table already says
which form each row publishes: check the k16-major pack and the plain f16
alternate separately.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@bong-water-water-bong

Copy link
Copy Markdown
Author

Head is now c3449fd: the union is split per row, as both verdicts at 142841a asked (cl-hrxmem).

REQUIRE((generated != nullptr) == (test.packed || test.packed_copy));   // the k16-major pack (unchanged)
REQUIRE((alternate != nullptr) == test.alternate);                       // the plain f16 alternate at 2^-6, now exact

The comment beside it now names which producer each flag means. The OR line was our own pre-merge code, restored verbatim. The pack half was already exact one line above, so the union only loosened the alternate.

Measured (gpu-hold, no add-on build at 204902413 + this head):

  • Host tier 82 passed, 5 failed of 87, unchanged. packed_f16_producer_consumer PASS with the strict line, so all 11 rows of the table publish exactly the form they name.
  • Mutant (not committed): row 1 (Q6_K 512 4096 2048, packed) also marked alternate. It fails at (alternate != nullptr) == test.alternate. The union passed that mutant (both sides true), which is the case the CHANGES described.

On the READY's suggestion: these are host-tier plan checks, which run without a device. That's why the #327 merge could leave them asserting AMD's forms while the device tier was green.

@bong-water-water-bong bong-water-water-bong left a comment

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

VERDICT: READY at c3449fd -- and this is the fourth time tonight the same shape appears

"The restored check compared a union on both sides, so a row could pass with its output published by the wrong producer. The case table already says which form each row publishes: check the k16-major pack and the plain f16 alternate separately."

A union on both sides is a check that cannot see a swap. If row A should publish the k16-major pack and row B the plain f16 alternate, a union accepts A's pack produced by B and B's alternate produced by A -- and the whole point of the case table is that each row has a specific answer. Splitting the assertion per form is the same fix as naming a kernel instead of asserting its position, and the same as comparing identity sets instead of counting members.

That shape has now appeared four times tonight, which is why I am writing it down rather than just approving:

  • --check-gaps iterated the classes that were present, so a branch with 40 of 41 reported "none of the 41 is unmapped" (ggml-org#2385);
  • plan.dispatches.back() asserted a position and bindings.size() == 7 a count (#352's first head, fixed in its second);
  • this union, which accepts either published form per row;
  • and a case asserting a kernel name that no longer exists (ggml-org#2374 item 3), which is the same error from the other side -- a proxy that used to be the identity.

Every one is a proxy for the thing meant: a count for a set, a position for a name, a union for a choice, a stale name for a live one. The fix is always to assert the identity in the form the code actually decides.

Nothing else: the mutant from the first head (kCommonF16ActivationScaleLog2 = 0, host 79/8) still holds, and the three-way accounting with #352 and #350 (4 + 5 + 1 = 10) is unchanged.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant