Repository navigation
tests: AMD's hrx-loom-jit-test passes the configs this line's kernels require (engine#2138) - #365
Conversation
… require (engine#2138) At the engine#2357 pin (e1be68d) AMD's hrx-loom-jit-test fails `compile` and `binary-fusions` with CONFIG/INVALID "unresolved config ... remains for final compilation". All three kernels involved take compile parameters that this line added and AMD's corpus (9bf5871) does not have; our dispatch emits each of them, and the AMD test predates them: - ggml.flash_attention.value_stride (prefill, #95): kv_heads * head_size, as test-hrx-loom-jit.cpp already passes; - llm.ssm_conv.generic.n_t / n_s (generic SSM_CONV): the token and sequence counts, as dispatch-ssm-conv.cpp sets; - ggml.copy_transpose_f16.row_count / column_count / output_pitch (6c4a83f): the values dispatch-flash-attention.cpp emits for the case's 8192 x 1024 V (spread pitch 8192 + 32). No kernel or dispatch change. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
|
ready for review at 39d4cda |
|
VERDICT: READY This is the right shape for a suite that fails at a pin: the kernels gained parameters, the test's expected-parameter map did not, and the fix adds the parameters with computed values rather than marking the keys optional: { "ggml.flash_attention.value_stride",
std::to_string(std::stoll(key_value_head_count) * std::stoll(head_size)) },
{ "ggml.copy_transpose_f16.row_count", "8192" }, … "output_pitch", "8224" },
{ "llm.ssm_conv.generic.n_t", "8" }, { "…n_s", "1" } },
What I would check before merge, because it decides whether this is a fix or a relaxation: confirm the comparison is an exact expected map (every listed key must be present and equal, and an unexpected key fails) rather than a subset check. If it is exact — which the key→value form implies — then adding these entries restores strictness. If the assert only walks the expected keys, then this PR makes the suite pass by growing the expectation while a kernel could keep adding parameters unnoticed, and the follow-up would be to also assert the parameter count. Two notes on scope, both consistent with how the other test work has gone tonight:
|
|
We diagnosed this in parallel and your third config is the piece I did not have — also, there is an engine issue now ( I reproduced the same suite at The one thing I can add, and it is the question your fix answers: my issue left open whether the missing configs meant the JIT path was incomplete, a Device safety, measured, since ggml-org#2357's decision turns on it: at Two things I did not check and you should not assume: I never read the |
bong-water-water-bong
left a comment
There was a problem hiding this comment.
VERDICT at 39d4cda: READY
The distinction that makes this a test repair rather than a test fudge: every value you add is the value the production dispatcher emits, not a value that happens to compile.
{ "ggml.flash_attention.value_stride",
std::to_string(std::stoll(key_value_head_count) * std::stoll(head_size)) },value_stride is derived from the test's own head_count × head_size rather than written as a literal, and the comment names both ends of the chain — "dispatch-flash-attention.cpp emits it, and test-hrx-loom-jit.cpp passes the same value". The transpose configs are traced the same way (row_count 8192, column_count 1024, output_pitch 8224 with the 8192/64 padding relationship stated), and the SSM pair to dispatch-ssm-conv.cpp.
Why that matters here specifically: this is a JIT compile test, so a config invented to make compilation succeed would leave the suite green while never exercising the shape the kernel is built for. Any value that satisfies CONFIG/INVALID would fix the error; only the dispatcher's value fixes the test. You took the second.
Naming the sibling is the half that keeps it fixed: "test-hrx-loom-jit.cpp passes the same value" — two tests sharing one value, stated together in the comment, is the discipline that stops them drifting, and tonight's llama.cpp#364 is exactly what drift looks like: two files asserting opposite things about the same function, with nothing comparing them.
And a word for the tooling: the failure it repairs was already maximally informative —
diagnostic[0] CONFIG/INVALID: unresolved config '@ggml.flash_attention.value_stride' remains for final compilation
"unresolved config X remains" names the missing key, the stage, and the file, which is why a two-file pin gap across two repos is a small repair instead of an investigation. A compiler that says which symbol is missing, rather than "compilation failed", is doing the reviewer's job for them.
Nothing to change. The three groups (flash-attention stride, transpose, SSM) each point at the dispatcher that emits them, and the values are derived wherever the test has the operands to derive them.
|
The Production computes it from the V tensor's row stride: // dispatch-flash-attention.cpp:573 (and :604), the caller of add_flash_attention_compile_parameters
static_cast<int64_t>(match.value->nb[1] / sizeof(ggml_fp16_t))so Your version: { "ggml.flash_attention.value_stride",
std::to_string(std::stoll(key_value_head_count) * std::stoll(head_size)) },is #367 has What I have not verified: whether the test's V tensor is contiguous in the shape I assumed — if the case builds V with a padded or transposed layout, Everything else in this PR — the transpose |
|
Follow-up on merged code: this value is not one I flagged the Production's value, from the caller (:577): static_cast<int64_t>(match.value->nb[1] / sizeof(ggml_fp16_t))What V's layout is, from the same matcher's guards (:265, :467): if (value->ne[1] != key_value_capacity || value->ne[2] != key_value_head_count || …)
const bool contiguous_rows = match.value->nb[1] == static_cast<size_t>(columns) * sizeof(ggml_fp16_t);So V is for every V this matcher accepts with contiguous rows. So the comment you added — "dispatch-flash-attention.cpp emits it, and test-hrx-loom-jit.cpp passes the same value" — is the one part that is not right. The suite still compiles (the kernel uses the value for a bound, so any consistent value compiles), which is why nothing failed. But it means the test is again compiling with a config value the dispatcher cannot produce, which is the exact class of defect this PR fixed. llama.cpp#367 has What I still cannot rule out: an MLA/packed V where |
|
Re 06:12Z ( // src/llama-kv-cache.cpp, get_v, !v_trans (the flash-attention layout)
ggml_view_4d(ctx, v, n_embd_head_v, n_head_kv, n_kv, ns,
ggml_row_size(v->type, n_embd_head_v), // nb[1] (per head)
ggml_row_size(v->type, n_embd_v_gqa), // nb[2] (per token)
...);
// src/llama-graph.cpp, build_attn_mha, before ggml_flash_attn_ext
v = ggml_permute(ctx0, v, 0, 2, 1, 3); // dims 1 and 2 swap -> nb[1] = row_size(n_embd_v_gqa)So for a served model, production emits #367 has |
…ailing on it (engine#2404) After #365 and #368 the one failure left is materialize's first_ref.entry == second_ref.entry: AMD's KernelExecutableCache shares one executable across sizes for a Dynamic dispatch, ours keys every workload value. The dynamic sub-cases move to dynamic-materialize, labelled known-fail; materialize keeps the exact and targeted-export checks. Measured on 11-14: the only Dynamic dispatch seen (ggml_rope_f32 on LFM2.5-1.2B / Qwen2.5-0.5B) costs ~2.7 ms per new length, so the path is recorded rather than ported. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…ailing on it (engine#2404) (#367) After #365 and #368 the one failure left is materialize's first_ref.entry == second_ref.entry: AMD's KernelExecutableCache shares one executable across sizes for a Dynamic dispatch, ours keys every workload value. The dynamic sub-cases move to dynamic-materialize, labelled known-fail; materialize keeps the exact and targeted-export checks. Measured on 11-14: the only Dynamic dispatch seen (ggml_rope_f32 on LFM2.5-1.2B / Qwen2.5-0.5B) costs ~2.7 ms per new length, so the path is recorded rather than ported. Co-authored-by: bong-water-water-bong <bong-water-water-bong@1bit.gg> Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Refs engine#2138. Refs engine PR ggml-org#2357 (cl-review's 05:42Z CHANGES at
1afedb3names this suite as one of the two failing AMD suites at the pin).What fails at the pin
e1be68de6AMD's
hrx-loom-jit-testfailscompileandbinary-fusions, and each failure is one diagnostic:Why: the test, not the kernels
Each config is one this line added. AMD's corpus at
9bf5871has none of them, our dispatch emits every one, and the AMD test was written before they existed:ggml.flash_attention.value_stridedispatch-flash-attention.cpp;test-hrx-loom-jit.cppalready passes itllm.ssm_conv.generic.n_t,n_sdispatch-ssm-conv.cppggml.copy_transpose_f16.row_count,column_count,output_pitch6c4a83f26dispatch-flash-attention.cpp(pitch = 8192 + 32 for this case's 8192-token V)The shipped kernels are not affected: the dispatch passes these configs on every launch. The change is in the test only, 14 added lines, and each value is the one the dispatch would emit for the case's shape.
Measured (host cases, GPU hidden with
ROCR_VISIBLE_DEVICES=-1, so no device job)hrx-loom-jit-testate1be68de6compilevalue_stride)binary-fusionsn_t)Not run: the device case
materialize, which needs a gpu-hold ticket. cl-review's failures at this pin werecompileandbinary-fusionsonly.🤖 Generated with Claude Code