perf(GFX1100-TG200): T14 row-split greedy argmax arm - #2876
Conversation
8ea44eb to
6dba6cc
Compare
… rewrote Eleven of this campaign's pull requests are siblings on the same base and each rewrites `## Now` whole, so merging them is last-writer-wins on that section and three records did not survive the sequence. This restores them against the landed tree rather than against any one branch's copy. - The T34 evidence link comes back out. That file is on no branch and not on main, so check-agent-record reads it as a dangling link and reds the agent-record job. The capture itself is stated in the prose; only the file backing it is missing, so it becomes a named owed item (#2936). - VT_ARGMAX_SPLIT is named as T14's lever. #2876 owed that record and its own copy of `## Now` was overwritten by a later sibling. - `## Owed` gains the missing T34 capture. The T37 closed-negative from #2892 and the T35/T35-r3 records from #2893 survived the sequence and are unchanged here. This is the shape AGENTS.md names: a single file every pull request must write is a lock, and this section is one. It is reconciled once, at the end, instead of eighteen times. Refs #2936. FOLLOWING_AGENTS_PROTOCOL Following-Agents-Protocol: true AI-Assisted: true Assisted-by: AGENT:claude-opus-5-1m [claude-code]
…overwrote Eleven of this campaign's pull requests are siblings on one base and each rewrites the spec's `## Now` whole, so merging them is last-writer-wins on that section. Four records did not survive the sequence; this restores them against the landed tree rather than against any one branch's copy. - The T34 and T35 evidence links come out. Neither file is on any branch or on main, so check-agent-record reads them as dangling and reds the agent-record job. Both measurements survive in prose: T34's capture is stated in full in `## Now`, and T35's round-3 file carries the round-2 provenance, including that the 7415e281 divergence was measured under the corrupted Q6_K MMVQ arm and does not survive the fixed kernel. T34's missing capture becomes a named owed item. - VT_ARGMAX_SPLIT is named as T14's lever. #2876 owed that record and a later sibling overwrote its copy of `## Now`. - The T2b evidence file keeps its closed negative. #2777 deleted the paragraph to drop a stale "hindsight unavailable" preamble and took the finding with it: the pointer-keyed activation-quant cache is unsound under the DevicePool block-recycling allocator. AGENTS.md does not allow deleting evidence to reduce context, so the finding is restated without the preamble. The T37 closed-negative from #2892 and the T35-r3 record from #2893 survived and are unchanged. A single file every pull request must write is a lock, which is what this spec section became. It is reconciled once, at the end, instead of eighteen times. Refs #2936. FOLLOWING_AGENTS_PROTOCOL Following-Agents-Protocol: true AI-Assisted: true Assisted-by: AGENT:claude-opus-5-1m [claude-code]
|
Held back with the rest of the I reviewed this change on its own and have no objection to it. Once #2790's base Landing today from this set: #2782 (with the grouped-Q8_0 repair), #2777 and |
6dba6cc to
9295338
Compare
|
Rebased onto the repaired stack tip 2ff6af4 (base b9f2ef4, the external-contributor landing branch). Gates green (check-env-doc, check-agent-record, check-rocm-dp4a-intrinsic); vllm-cli + test_rocm_quant_dot compile and link at -Werror in rocm-dev:10.0.0 on gfx1100; the default path (all arms off) is byte-identical to the staging baseline on Qwen3.5-4B Q4_K_M (canonical TG200 prompt, 256 tokens, greedy, seed 0). Engine A/B numbers measured today (one window, 5 reps, one load per arm): default 48.4 tok/s median; VT_GEMV_MMVQ=1 alone 53.5 (+10.0%); the byte-exact arm group (VT_NORM_QUANT_FUSED, VT_QUANT_Q8K_WARP) verified token-identical solo; VT_SKINNY_BF16 and the keep-quant arms (VT_GDN_ROWPERM_KEEP_QUANT / VT_GDN_COLPERM_KEEP_QUANT) and the association-change arms (VT_ATTN_DECODE_GQA4, VT_RMSNORM_ROW_COOP) move early near-tie tokens on the engine and are NOT engine-token-identical — flagged here so the acceptance terms name them; the full opt-in stack measured 72.5 tok/s median (+49.6%) with that documented divergence. This branch's head is now 9295338. Ping for re-review. FOLLOWING_AGENTS_PROTOCOL Following-Agents-Protocol: true |
bb3bbe1 to
4411e5e
Compare
|
Rebased onto FOLLOWING_AGENTS_PROTOCOL Following-Agents-Protocol: true |
e1ffd6e to
298523c
Compare
The donor argmax launches one block per row: batch-1 decode walks the full 248,320-element vocab with ~970 serial load+compare iterations per lane — 153.96us against a ~2-3us memory floor. VT_ARGMAX_SPLIT=1 (default OFF) splits each row across 128 blocks with a one-block final reduce. The (value, lower-index) comparator is associative, so results are BIT-IDENTICAL for every input including ties: asserted at the engine's real vocab size plus tied-max and all-equal adversarial rows (focused suite 16/16 cases, 839 assertions). Engagement capture shows both phase kernels at 34.2+10.5us = 44.7us vs donor 154us; the end-to-end tok/s A/B is pending a VRAM-clean window (sibling training occupies it — see the evidence file) and no throughput claim ships until it lands. FOLLOWING_AGENTS_PROTOCOL Following-Agents-Protocol: true AI-Assisted: true Assisted-by: AGENT:ox-alpha [omp] Assisted-by: AGENT:OMEN-ALPHA [OMP]
…es the arm The T14 arm cached the flag in a function-local static, and the byte- identity test's first GreedyArgmax call runs the flag-OFF leg — so the flag-ON leg silently reused the serial reduction and the split kernels under test never executed (the test passed vacuously; a mutation to the Phase-B tie-break stayed green). Read the flag per call like the CUDA sibling's convention (cuda_quant_dot.cu:1006), matching every other VT_ lever in this stack; the workspace statics stay, since they cache capacity, not the switch. FOLLOWING_AGENTS_PROTOCOL Following-Agents-Protocol: true AI-Assisted: true Assisted-by: AGENT:GLM-5.3-Flash [ZCode]
298523c to
8082ede
Compare
2 participants
Closes #2873.
Row:
GFX1100-TG200T14 adds a row-split greedy argmax arm, opt-in behind
VT_ARGMAX_SPLIT=1(default OFF). The row-split kernel distributes the argmax reduction across
warps.
A/B benchmark (interleaved, 5 pairs, Qwen3.5-4B Q4_K_M, 256 tokens)
+1.0% over T25 parent. Small but consistent (stdev 0.15%). Token-identical.
Depends on the full T25 chain: #2782, #2790, #2792, #2796, #2800, #2804, #2807.
FOLLOWING_AGENTS_PROTOCOL
Following-Agents-Protocol: true
AI-Assisted: true
Assisted-by: AGENT:GLM-5-2 [OMP]