Repository navigation
tests: fa_prefill_full_cache checks the merged mask path, not the retired compact mask (engine#2374 item 3) - #343
Conversation
…ired compact mask (engine#2374 item 3) Since the engine#2138 merge (#327) the postprocess match no longer publishes a compact [tokens x tokens] F16 mask through a qwen_attention_metadata initialisation dispatch; AMD removed both the code and this case at 9bf5871. The flash attention reads the graph's own mask over the whole key/value view, one query chunk at a time. The case now checks that path: no metadata initialisation, every FA chunk at key_value_token_count = the view, binding 3 = the graph mask at the chunk's own row offset and length, and the chunks covering every query token. A real graph's view is get_n_kv() (used cells padded to 256), so the full-capacity view only occurs in the reserve graph or a full cache. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
|
VERDICT: READY at What the change actually buysThe case was written against the pre-merge design, so after #327 it asserted something the code no longer does — a stale test is worse than a missing one, because it either fails for the wrong reason or gets loosened until it passes. Three parts of the update matter more than they look:
One number in the comment is doing two jobs
The question I would ask before merging, and the answer I foundIt is a scheduling case ( Worth one clause in the body saying that, because the obvious reviewer question — "you changed what this test checks; who checks the numbers now?" — has a good answer and should not require the next reviewer to run the same grep. |
|
VERDICT: READY This is a rewrite of a case's expectation (+25/−25 in a test), which is the shape I normally challenge — and it holds up, for reasons worth naming. It flips "the old design is present" into "the old design is absent, and the new one is positively required": The prohibition is an active assertion, not a deleted one: if the pre-ggml-org#2138 metadata dispatch ever comes back into this plan, the case fails. That is the right way to retire an expectation — keep the old behaviour as a named thing you refuse, rather than silently dropping the line that mentioned it. And the new assertions are precisely shaped, not "a dispatch exists": The offset/length pair is the part that actually proves the claim in the PR description — each query chunk binds its own rows of the full-capacity mask — and it is checked against a running The conditional loop is guarded, which is the failure I would most expect here: the One thing to confirm before merge (not a blocker for me, a check for you): the case now depends on |
Refs engine#2374, item 3 (
fa_prefill_full_cache.512: "the compact F16 mask alternate is absent from the plan"). Items 1 and 2 are not touched (1 is #341; 2 is pi-i679's FA accumulator).What the case was checking
The case asserted the pre-merge design: the postprocess match published a compact
[tokens x tokens]F16 mask as an alternate value, through aqwen3_moe:qwen_attention_metadatainitialization dispatch, and the flash attention ran atkey_value_token_count == token_countagainst that compact mask. The engine#2138 merge (#327) took AMD'sdispatch-qwen-attention-postprocess.cpp, which no longer has that path (-153 lines:append_attention_metadata_initializationand theqwen.attention.compact_maskalternate). AMD removed the code and this case at9bf5871:git grep fa_prefill_full_cache 9bf5871 -- testsandgit grep attention_metadata 9bf5871 -- '*.cpp'are both empty. Our test file kept the case, so after the merge it checked a path that no longer exists.What the merged plan does (measured, gpu-hold, HRX0, base
e7b8a14db, plan printed by a local probe that is not in this PR)So the flash attention reads the graph's own mask over the whole key/value view, in query chunks, each chunk binding its own mask rows. On this case's full-capacity view (40,960 rows for a 512-token prefill), that is 80x the key/value rows of the old compact path. That cost does not reach a real graph.
llama_kv_cache::get_n_kv()sizes the view to the used cells padded to 256, andsrc/models/qwen4exp.cppuses the sameget_n_kv(). A full-capacity view occurs only in the reserve graph (kv->get_size()) or with a full cache, where every row is needed anyway. The compact mask saved at most the <256 padding rows.The change (test only)
fa_prefill_full_cache.512now checks the merged path end to end:qwen_attention_metadatainitialization dispatch;key_value_token_count= the view (mask->ne[0]) and has 6 bindings withapply_gate == "0";first_query * kv * 2) and length (chunk_tokens * kv * 2);Verified (strixhalo gfx1151, gpu-hold, no add-on build at
e7b8a14db+ this commit)test-hrx-ops --hrx-ops-test-test-filter '^(fa_|flash_attention)': 35 passed, 1 failed, 36 total.fa_prefill_full_cache.512PASS. The one failure isflash_attention.accumulator_range.128.1024.v60000, which is item 2: it is in the issue's failure list for the base, and I did not rerun the base myself.dispatch-query-chunks.cpp, not committed). The case fails atbindings[3].offset == covered_query_tokens * mask_row_bytes, so the new offset check fires on a wrong chunk mask.tests/test-hrx-ops.cppchanges, in this one case.If the compact mask should come back as an optimization instead, that would be porting the 153 lines back onto AMD's file; given
get_n_kv(), I don't think it buys anything on a real graph.🤖 Generated with Claude Code