Skip to content

feat(KERNEL-QUANT-CIQ-GEMM-ROCM-RDNA3): enable gfx1100 WMMA prefill - #3187

Open
VikashLoomba wants to merge 5 commits into
mudler:mainfrom
VikashLoomba:row/KERNEL-QUANT-CIQ-GEMM-ROCM-RDNA3
Open

feat(KERNEL-QUANT-CIQ-GEMM-ROCM-RDNA3): enable gfx1100 WMMA prefill#3187
VikashLoomba wants to merge 5 commits into
mudler:mainfrom
VikashLoomba:row/KERNEL-QUANT-CIQ-GEMM-ROCM-RDNA3

Conversation

@VikashLoomba

@VikashLoomba VikashLoomba commented Sep 14, 2026

Copy link
Copy Markdown
Contributor

Enable Q4_K and Q6_K WMMA prefill on gfx1100 through the existing quantized matmul dispatch. Both tile bodies remain byte-identical; the change adds quant-specific architecture admission. Attention retains its separate gate, and VT_ROCM_QUANT_WMMA=0 retains the scalar control.

Row

KERNEL-QUANT-CIQ-GEMM-ROCM-RDNA3; resolves ISSUE-LOCAL-01M2F0PQWGSCXG0N4951NF9DPZ.

Before starting

The current source at cef9f821632eb33156bb7d24a0a85eeab7f7484a excludes gfx1100 from the quantized WMMA compile and runtime gates. The existing physical tests could skip that device. The issue/PR and ownership search found no existing gfx1100 admission change. The developer selected one PR for specification and implementation. Specification commit a5b5c92 precedes implementation c3fe98ba6.

Evidence

  • The gfx1100 red-first hardware run fails ten dispatch assertions before admission; numerical scalar checks still pass.
  • Four focused physical cases pass all 46 assertions, including Q4_K/Q6_K, F32/BF16, partial blocks and asymmetric tails.
  • The original upstream MMQ corpus passes all 240 cases against primary outputs and dense references with unchanged tolerances.
  • The public completion test checks both launches, all 1,024 logits and eight tokens against a separate scalar process. Removing either launch breaks output checks even with its counter retained.
  • The fresh reviewer reports PASS with no findings after architecture, launch, scalar-control and corpus-integrity mutations. The operator independently rebuilt and reran the hardware/public gates.
  • A separate final-record reviewer reports PASS with no findings at d6e40c91f634a041c873c7a04516d55c4d05772a. The review verifies receipt mappings and hashes, authored links, lifecycle records, and the explicit performance qualifications.
  • The broad HIP gate has 24 executed tests passing and five paired fixture/device skips. The architecture suite passes 16 cases and 109 assertions. The emitted gfx1100 object contains signed-int8 WMMA instructions.

Exact commands, hashes, red/green results, mutation restorations and review evidence are in docs/bench-evidence/rocm-rdna3-quant-wmma/README.md.

Preflight qualification

The implementer and reviewer full staged preflights exit zero with no failed checks and 12 explicit baseline skips. These are not all-green preflights. Seven NumPy-dependent suites subsequently run with an existing isolated NumPy installation; all exit zero, with five model-dependent CLIP subcases still skipped. Four uninvoked architecture checks concern unchanged ARM/CPU/CUDA/Triton paths. The final PR classification check runs separately with its required arguments.

The final operator agent-ready.py run at d6e40c91f634a041c873c7a04516d55c4d05772a reports zero failed checks and compiles all eight affected host translation units. It exits one because its strict preflight skips five argument-dependent checks: ARM ISA, CPU ISA, CUDA gencode, PR classification, and Triton AOT. PR classification passes separately locally and in CI. The other four concern unchanged architecture paths. This is a qualified result, not an all-green readiness claim.

The operator's initial coordinator branch name caused the role checker to treat pending work as landed. Renaming that task branch to the required row/ form resolves the check without changing source or the checker. Initial receipts and scoped resolutions remain preserved. These narrowly scoped baseline/resource qualifications do not waive the gfx1100 correctness, reachability or independent-review gates.

Model evidence and remaining work

The real-model workload uses two raw prompts of 183 and 174 tokens, repeated four times, with 16 greedy output tokens per request. All native input IDs match the primary tokenizer exactly. All 128 completion IDs match across six interleaved native on/off processes and both upstream runtimes. Matching native traces show 1,216 WMMA launches confined to eight prefill windows and none during 120 decode windows; the scalar control shows zero WMMA launches.

Fresh builds of vLLM main 39545e475d3627287ff69c25465dc0bd405f67e1 and llama.cpp master 093a2f86c3e37c54fa3e1f9efb17b304f3433abd execute the retained Qwen3.5-4B Q4_K_M workload; the GGUF plugin main remains d4c1f0d082fc7cd4350da56689109a01c1f29d6c.

Other gfx11 devices and attention WMMA admission remain excluded. Full-model performance floors remain failing under ISSUE-LOCAL-01M2F4WCD6ZK5VH5S8TF83APD6. Dynamic-clock timing observations and sampled memory records carry explicit limits; this PR does not claim accepted clock-attributed speedups, whole-model performance parity, or equal-cache-capacity memory efficiency. RDNA4 hardware was not rerun; its architecture admission and tile bodies are preserved.

FOLLOWING_AGENTS_PROTOCOL

Following-Agents-Protocol: true
AI-Assisted: true
Assisted-by: AGENT:gpt-6-astra [Codex]

The generic rocWMMA tile API already supplies RDNA3 operand transforms. Prove that admitting gfx1100 reuses the Q4_K and Q6_K bodies before changing arithmetic. Keep attention admission separate and compare fresh upstream snapshots requested by the developer.

Tracks ISSUE-LOCAL-01M2F0PQWGSCXG0N4951NF9DPZ. Staged preflight matches the pristine baseline: test_agent_onboard assumes master although global Git config selects main. Its 39 tests pass with GIT_CONFIG_GLOBAL=/dev/null. Twelve unrelated gates report explicit skips. Record and staged-diff checks pass.

FOLLOWING_AGENTS_PROTOCOL

Following-Agents-Protocol: true
AI-Assisted: true
Assisted-by: AGENT:gpt-6-astra [Codex]
…prefill

Reuse the existing Q4_K and Q6_K rocWMMA tiles on gfx1100. The fragment
API already resolves its operand layout. Keep attention admission separate.

Hardware red-first tests fail ten dispatch assertions before admission.
The unchanged kernels pass all four focused cases and 46 assertions after
admission. The scalar control passes the same cases in a separate process.
The public completion gate reaches both formats and matches all 1024 logits
and eight generated tokens against its scalar child process.

Port all 240 original plugin test_mmq Q4_K/Q6_K tensor cases, preserving
inputs and tolerances. Native outputs pass against both the plugin and its
dense reference. F16 output narrows from native F32 in the test adapter.
The gfx1100 code object contains signed-int8 WMMA instructions.

This is an immutable local implementation checkpoint. The staged preflight
is running. Broad HIP gates, mutation review, and full-model performance
remain pending. No performance claim or remote handoff is authorized by
this checkpoint. Tracks ISSUE-LOCAL-01M2F0PQWGSCXG0N4951NF9DPZ.

FOLLOWING_AGENTS_PROTOCOL

Following-Agents-Protocol: true
AI-Assisted: true
Assisted-by: Codex:GPT-6 [Codex]
…receipts

Keep the physical red-first results, original primary fixture comparisons,
public scalar control, emitted instructions, and negative mutations with
the architecture admission change. Both executing tile bodies retain their
committed-spec hashes.

The complete HIP gate passes all 29 registered tests with five unchanged
resource skips. The full staged preflight exits zero with 12 unchanged
skips. Four CPU and four public production mutations fail as intended,
with immutable source hashes preserved. Full-model performance and fresh
review remain with the coordinating operator.

Tracks ISSUE-LOCAL-01M2F0PQWGSCXG0N4951NF9DPZ.

FOLLOWING_AGENTS_PROTOCOL

Following-Agents-Protocol: true
AI-Assisted: true
Assisted-by: Codex:GPT-6 [Codex]
…ication

Preserve operator and fresh-review evidence for the unchanged implementation.
The review passes with no findings. The operator HIP gate passes with the
same five resource skips, and all seven supplemental NumPy suites execute.
Keep the original harness failures and unresolved model gate explicit.

Tracks ISSUE-LOCAL-01M2F0PQWGSCXG0N4951NF9DPZ.

FOLLOWING_AGENTS_PROTOCOL

Following-Agents-Protocol: true
AI-Assisted: true
Assisted-by: Codex:GPT-6 [Codex]
…ssion

Record exact model input and output identity, interleaved native results,
prefill-only WMMA traces, and the operator's 240 original matrix cases.
Close architecture admission with the reviewed implementation unchanged.
Keep every full-model floor and clock-attribution obligation open under
ISSUE-LOCAL-01M2F4WCD6ZK5VH5S8TF83APD6.

Flatten evidence into the classifier's permitted per-run layout. Preserve
source bytes and hashes, update copy maps, and reconcile only the parent's
gfx1100 record. No product, test, tool, or checker changes are included.

The full preflight exits 1 on the known onboarding fixture's branch-name
assumption and retains 12 skips. Its unchanged 39-case suite passes with
GIT_CONFIG_GLOBAL=/dev/null. Scoped record, link, hash, role, and staged
checks pass. Prior independent implementation review remains PASS.

Closes ISSUE-LOCAL-01M2F0PQWGSCXG0N4951NF9DPZ.
Closes ISSUE-LOCAL-01M2F51K16TDAC95DM1VEKBBCK.

FOLLOWING_AGENTS_PROTOCOL

Following-Agents-Protocol: true
AI-Assisted: true
Assisted-by: Codex:GPT-6 [Codex]
@VikashLoomba
VikashLoomba marked this pull request as ready for review September 14, 2026 05:29
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant