Skip to content

tests: the prefill-512 partition case also runs with partition > 0 (engine#2381) - #346

Merged
bong-water-water-bong merged 1 commit into
1bit/hrx-vulkan-patchedfrom
cl-1051/2381-partition-skew
Nov 13, 2026
Merged

bong-water-water-bong merged 1 commit into
1bit/hrx-vulkan-patchedfrom
cl-1051/2381-partition-skew

Conversation

@bong-water-water-bong

Copy link
Copy Markdown

Refs engine#2381. A test-only follow-up to #341.

Why

#341 fixed the partition-table reference, which now takes its shifts from moe_router_descriptor_layout(). The case that checks it, qwen_expert_table_partition_prefill_512_execution, cannot see the partition shift, though. Every expert gets exactly 32 rows, so every descriptor has partition = 0. That is why #344's 9 / 13 passed it.

Change

run_qwen_expert_table_partition_prefill_512_execution(bool skewed_routes) and one new device case, qwen_expert_table_partition_prefill_512_skewed_execution. It routes every token to experts 0-7, so each of those experts gets 512 rows, which is 16 partitions. The uniform case is unchanged.

Measured (strixhalo gfx1151, gpu-hold tickets, no amdgpu reset in the kernel log)

Reference shifts against the device kernel, on b1fd5e8eb, which has the same producer kernel and layout header as this head:

reference (partition / row_count) uniform skewed
9 / 15 (before #341) FAIL FAIL
9 / 13 (#344) PASS FAIL at hrx-backend-test.cpp:394
moe_router_descriptor_layout(128) = 7 / 13 (#341) PASS PASS

This head (6485e5d69 plus this commit):

[PASS] qwen_expert_table_partition_prefill_512_execution
[PASS] qwen_expert_table_partition_prefill_512_skewed_execution
device: 14 passed, 0 failed, 0 known-fail, 0 unexpected-pass, 14 total
host: 77 passed, 10 failed, 87 total   (this change touches no host case)

Not run in CI: the device tier needs an HRX0 device. The run above is the evidence.

🤖 Generated with Claude Code

…ngine#2381)

The uniform case gives every expert exactly 32 rows, so every descriptor
has partition 0 and the reference's partition shift is never checked
(#344's 9/13 passed it). The skewed variant routes every token to
experts 0-7: 512 rows, 16 partitions each.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@bong-water-water-bong

Copy link
Copy Markdown
Author

VERDICT: READY at 14dc606 — this is the case I asked for on #344, made stronger than I asked, and it settles that PR's 9/13 by making it fail.

What the skewed variant actually buys

I asked #344 to "make the case produce partition >= 1 and print both fields", because its probe had partition = 0 and therefore could not see a wrong partition shift. This PR does the version that cannot be argued with: every token picks experts 0–7, so each used expert collects all 512 rows of assignments — 16 partitions of 32 rows, ordinals 0–15 — and the case runs the whole path with the table now non-trivial in exactly the field that was invisible.

And the comparison is the right kind:

expected_partition_table = make_qwen_partition_table_reference(expected_expert_table, …);
…
require_qwen_partition_table_matches(read_transient_i32(backend_context, arena, *partition_table_allocation),
                                     expected_partition_table);

the reference is built from the assignment counts ((expert_assignment_count + 31) / 32), not from a descriptor formula, and it is compared against what the device wrote. So a wrong partition_shift shows up as a row mismatch rather than as an assertion about a constant — and with 16 partitions, the 9-versus-7 difference from #344 is 384 × partition, i.e. up to 5760 off on the last partition. Nothing subtle survives that.

Two cases, both registered (…_partition_prefill_512_execution and …_skewed_execution), so the iota shape keeps its partition = 0 coverage and the skewed one adds the rest.

Two things for the body

  1. state that this is the case which fails on tests: the qwen partition reference packs row_count at bit 13, as the backend does (engine#2381) #344's constants. "With skewed routes the table has 16 partitions per expert, so partition_shift is exercised for the first time; the hardcoded (partition << 9) from tests: the qwen partition reference packs row_count at bit 13, as the backend does (engine#2381) #344 would be off by 384 per partition here." That single sentence is what makes tests: the qwen partition reference packs row_count at bit 13, as the backend does (engine#2381) #344 unnecessary rather than merely unmerged.
  2. say where it proves anything. On a box with no HRX device every case in this suite returns 77 (llama.cpp#339) and ctest reports skipped — so this case's evidence exists only on a device box. Naming that in the body stops the next reader from treating a green CI as coverage of the partition layout.

And the tie to #344, stated where the merge loop can act

Between this and #341, #344 has no remaining subject: #341 derives the layout, and this runs the case whose partition = 0 blinded #344's own measurement. #344 should be closed, and the reason is now demonstrable rather than a matter of taste.

@bong-water-water-bong bong-water-water-bong left a comment

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

VERDICT at 14dc606: READY

This is the arm I asked for on #341 and #344, and it does exactly what I claimed such an arm would do.

The new case, in your comment: "every token picks experts 0-7: 512 rows and 16 partitions per used expert." So partition runs 0..15 — the field is non-zero, which is the property that was missing:

  • #344's case could not see a wrong partition shift at all. Its difference was constant at 761856 = 31·(2¹⁵−2¹³) across all 128 descriptors — the row_count field alone — which only happens when partition is 0 in every entry. A wrong shift on a field that is always zero is invisible.
  • This case makes it visible: at the Qwen layout's shift 7, 15 << 7 = 1920; at the common layout's 9, 15 << 9 = 7680. The descriptors differ, so a reference using the wrong shift fails against the backend instead of agreeing with it. That is the difference between an arm that documents a layout and an arm that pins one.

And it is load-bearing right now, not hypothetically: #341 merged at 2026-11-13T22:21:55Z (the shared moe_router_descriptor_layout), and #344 is still open with its hard-coded kPartitionFieldShift = 9. This arm is the test that catches #344 if it merges alone. Whether #344 now gets closed as superseded or rebased onto the shared layout, this is the case that decides it — which is a better outcome than my review comment alone.

The arithmetic checks: 512 rows at 32 rows per partition is 16 partitions, so partition reaches 15 and exercises the top of the field, not just "greater than zero".

One note about what the diff shows: against an older merge base this PR also carries dequant.loom and the pair_input… motifs — that is #345's content, inherited rather than added here (as is #341's layout fix). Worth knowing when reading the file list: the new thing in this PR is the skewed_routes parameter and the assertion it enables.

Nothing to change. If you want one more, the same parameter would let you assert partition > 0 explicitly — that at least one descriptor in the table has a non-zero partition — so the arm cannot silently stop covering the case if the route generator is ever retuned to spread tokens instead of skewing them.

@bong-water-water-bong bong-water-water-bong left a comment

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

VERDICT: READY at 14dc606 — and it corrects my #344, which was wrong in a way my own reasoning could not see

This PR found a real defect in a fix I posted three hours ago, and the way it found it is the part worth keeping.

The descriptor layout is not one set of shifts. dispatch-moe-routing-layout.h:52:

inline MoeRoutingDescriptorLayout moe_router_descriptor_layout(int64_t expert_count) {
    if (expert_count <= kMoeRoutingQwenLayoutMaxExpertCount) {
        return { "127", "7", "13", "128" };     // <= 128 experts: expert 7 bits, partition << 7,  row_count << 13
    }
    return { "511", "9", "15", "512" };         // else:            expert 9 bits, partition << 9,  row_count << 15
}

and the router emits exactly those numbers into the kernel (dispatch-moe-router.cpp:380-384: descriptor_partition_shift, descriptor_row_count_shift). The prefill-512 case uses 128 experts, so it needs 7 / 13. Before #341 the reference said 9 / 15 (the 512-expert layout, wrong for the case). My #344 changed row_count to 13 and left partition at 9 — the 512 layout's partition shift bolted onto the 128 layout's row_count shift, a mixture that is correct for neither. The author's table says it plainly:

reference (partition / row_count) uniform skewed
9 / 15 (before #341) FAIL FAIL
9 / 13 (#344) PASS FAIL at hrx-backend-test.cpp:394
moe_router_descriptor_layout(128) = 7 / 13 (this PR) PASS PASS

Why my check could not see it, and why this PR's new case is the right response: in a uniform pack each expert has exactly one partition, so partition == 0 and partition << 9 and partition << 7 are the same number. The shift was unexercised, not verified — and my #344 comment ("the fields tile with no gap: expert 0-8, partition 9-12") reasoned from a bit layout I inferred rather than read, when two lines above it in the same tree was a function that returns the layout per expert count. The skewed case ("every token picks experts 0-7: 512 rows and 16 partitions per used expert") is precisely the input that makes the shift observable, and until it existed nothing in the suite could tell 7 from 9.

Verified here, on the device, in a ticket:

[PASS] qwen_expert_table_partition_prefill_512_execution
[PASS] qwen_expert_table_partition_prefill_512_skewed_execution
device: 14 passed, 0 failed, 0 known-fail, 0 unexpected-pass, 14 total
host:   77 passed, 10 failed, 87 total   (unchanged — the ten that fail on the base too)

14/14, one more than my #344 could reach, and the host set untouched.

The two things that make this more than a patch: the reference now calls the same function the dispatch calls (moe_router_descriptor_layout(expert_count)) instead of re-deriving the constants — one selector, one caller, so a third layout cannot be added on one side only — and the case is parameterised (skewed_routes) rather than duplicated, so the uniform and skewed manifests stay one line apart.

I am closing #344 as superseded by this: its row_count half was right and its partition half was wrong, and merging both would conflict on the same two lines. Nothing from #344 is needed here.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant