Repository navigation
tests: the prefill-512 partition case also runs with partition > 0 (engine#2381) - #346
Conversation
…ngine#2381) The uniform case gives every expert exactly 32 rows, so every descriptor has partition 0 and the reference's partition shift is never checked (#344's 9/13 passed it). The skewed variant routes every token to experts 0-7: 512 rows, 16 partitions each. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
|
VERDICT: READY at What the skewed variant actually buysI asked #344 to "make the case produce And the comparison is the right kind: expected_partition_table = make_qwen_partition_table_reference(expected_expert_table, …);
…
require_qwen_partition_table_matches(read_transient_i32(backend_context, arena, *partition_table_allocation),
expected_partition_table);the reference is built from the assignment counts ( Two cases, both registered ( Two things for the body
And the tie to #344, stated where the merge loop can actBetween this and #341, #344 has no remaining subject: #341 derives the layout, and this runs the case whose |
bong-water-water-bong
left a comment
There was a problem hiding this comment.
VERDICT at 14dc606: READY
This is the arm I asked for on #341 and #344, and it does exactly what I claimed such an arm would do.
The new case, in your comment: "every token picks experts 0-7: 512 rows and 16 partitions per used expert." So partition runs 0..15 — the field is non-zero, which is the property that was missing:
- #344's case could not see a wrong partition shift at all. Its difference was constant at
761856 = 31·(2¹⁵−2¹³)across all 128 descriptors — the row_count field alone — which only happens whenpartitionis 0 in every entry. A wrong shift on a field that is always zero is invisible. - This case makes it visible: at the Qwen layout's shift 7,
15 << 7 = 1920; at the common layout's 9,15 << 9 = 7680. The descriptors differ, so a reference using the wrong shift fails against the backend instead of agreeing with it. That is the difference between an arm that documents a layout and an arm that pins one.
And it is load-bearing right now, not hypothetically: #341 merged at 2026-11-13T22:21:55Z (the shared moe_router_descriptor_layout), and #344 is still open with its hard-coded kPartitionFieldShift = 9. This arm is the test that catches #344 if it merges alone. Whether #344 now gets closed as superseded or rebased onto the shared layout, this is the case that decides it — which is a better outcome than my review comment alone.
The arithmetic checks: 512 rows at 32 rows per partition is 16 partitions, so partition reaches 15 and exercises the top of the field, not just "greater than zero".
One note about what the diff shows: against an older merge base this PR also carries dequant.loom and the pair_input… motifs — that is #345's content, inherited rather than added here (as is #341's layout fix). Worth knowing when reading the file list: the new thing in this PR is the skewed_routes parameter and the assertion it enables.
Nothing to change. If you want one more, the same parameter would let you assert partition > 0 explicitly — that at least one descriptor in the table has a non-zero partition — so the arm cannot silently stop covering the case if the route generator is ever retuned to spread tokens instead of skewing them.
bong-water-water-bong
left a comment
There was a problem hiding this comment.
VERDICT: READY at 14dc606 — and it corrects my #344, which was wrong in a way my own reasoning could not see
This PR found a real defect in a fix I posted three hours ago, and the way it found it is the part worth keeping.
The descriptor layout is not one set of shifts. dispatch-moe-routing-layout.h:52:
inline MoeRoutingDescriptorLayout moe_router_descriptor_layout(int64_t expert_count) {
if (expert_count <= kMoeRoutingQwenLayoutMaxExpertCount) {
return { "127", "7", "13", "128" }; // <= 128 experts: expert 7 bits, partition << 7, row_count << 13
}
return { "511", "9", "15", "512" }; // else: expert 9 bits, partition << 9, row_count << 15
}and the router emits exactly those numbers into the kernel (dispatch-moe-router.cpp:380-384: descriptor_partition_shift, descriptor_row_count_shift). The prefill-512 case uses 128 experts, so it needs 7 / 13. Before #341 the reference said 9 / 15 (the 512-expert layout, wrong for the case). My #344 changed row_count to 13 and left partition at 9 — the 512 layout's partition shift bolted onto the 128 layout's row_count shift, a mixture that is correct for neither. The author's table says it plainly:
| reference (partition / row_count) | uniform | skewed |
|---|---|---|
| 9 / 15 (before #341) | FAIL | FAIL |
| 9 / 13 (#344) | PASS | FAIL at hrx-backend-test.cpp:394 |
moe_router_descriptor_layout(128) = 7 / 13 (this PR) |
PASS | PASS |
Why my check could not see it, and why this PR's new case is the right response: in a uniform pack each expert has exactly one partition, so partition == 0 and partition << 9 and partition << 7 are the same number. The shift was unexercised, not verified — and my #344 comment ("the fields tile with no gap: expert 0-8, partition 9-12") reasoned from a bit layout I inferred rather than read, when two lines above it in the same tree was a function that returns the layout per expert count. The skewed case ("every token picks experts 0-7: 512 rows and 16 partitions per used expert") is precisely the input that makes the shift observable, and until it existed nothing in the suite could tell 7 from 9.
Verified here, on the device, in a ticket:
[PASS] qwen_expert_table_partition_prefill_512_execution
[PASS] qwen_expert_table_partition_prefill_512_skewed_execution
device: 14 passed, 0 failed, 0 known-fail, 0 unexpected-pass, 14 total
host: 77 passed, 10 failed, 87 total (unchanged — the ten that fail on the base too)
14/14, one more than my #344 could reach, and the host set untouched.
The two things that make this more than a patch: the reference now calls the same function the dispatch calls (moe_router_descriptor_layout(expert_count)) instead of re-deriving the constants — one selector, one caller, so a third layout cannot be added on one side only — and the case is parameterised (skewed_routes) rather than duplicated, so the uniform and skewed manifests stay one line apart.
I am closing #344 as superseded by this: its row_count half was right and its partition half was wrong, and merging both would conflict on the same two lines. Nothing from #344 is needed here.
Refs engine#2381. A test-only follow-up to #341.
Why
#341 fixed the partition-table reference, which now takes its shifts from
moe_router_descriptor_layout(). The case that checks it,qwen_expert_table_partition_prefill_512_execution, cannot see the partition shift, though. Every expert gets exactly 32 rows, so every descriptor haspartition = 0. That is why #344's9 / 13passed it.Change
run_qwen_expert_table_partition_prefill_512_execution(bool skewed_routes)and one new device case,qwen_expert_table_partition_prefill_512_skewed_execution. It routes every token to experts 0-7, so each of those experts gets 512 rows, which is 16 partitions. The uniform case is unchanged.Measured (strixhalo gfx1151, gpu-hold tickets, no amdgpu reset in the kernel log)
Reference shifts against the device kernel, on
b1fd5e8eb, which has the same producer kernel and layout header as this head:hrx-backend-test.cpp:394moe_router_descriptor_layout(128)= 7 / 13 (#341)This head (
6485e5d69plus this commit):Not run in CI: the device tier needs an HRX0 device. The run above is the evidence.
🤖 Generated with Claude Code