perf(GFX1100-TG200): T2b flips ROCm support_static_graph_mode

.agents/specs/gfx1100-tg200-t2b-static-graph-opt-in.md

@@ -0,0 +1,115 @@
+# Spec: GFX1100-TG200-T2b — per-arch opt-in for ROCm static-graph mode
+
+- Owning row: `GFX1100-TG200` (campaign issue #5, live landing `BACKEND-ROCM`
+  #2427). This spec does not create a new row; it reworks the T2b stage of the
+  existing row so the decode-graph flip is per-arch opt-in rather than
+  unconditional.
+- Pull request: #2777 (`row/GFX1100-TG200-T2b`).
+- Base: `b9f2ef432` (`upstream/stage/ext-prs-2026-09-04`, the external-contributor
+  landing branch).
+- Evidence: `docs/bench-evidence/gfx1100-tg200-t2b-20260823.md`.
+- Predecessor spec: `.agents/specs/rocm-decode-graph.md` (BACKEND-ROCM W1, the
+  hipGraph capture seam).
+
+## What the flip changes
+
+`RocmPlatform::support_static_graph_mode()` flips from the base-class `false`
+to `true`. This is the platform POLICY gate that, together with
+`Backend::SupportsGraphCapture()` and `vt::GraphCaptureEnabled()`, admits the
+`Qwen3_5DenseDecodeGraph` capture/replay path for uniform decode steps.
+
+The flip is NOT unconditional. Mirroring the Tenstorrent pattern landed in
+`src/vllm/platforms/tenstorrent.cpp` (#2910's `static_graph_requires_opt_in()`
+seam), `RocmPlatform` overrides both overloads of
+`static_graph_requires_opt_in()` so that only the evidence arch (gfx1100)
+gets default-on capture. Every other ROCm arch keeps the pre-flip opt-in
+polarity (capture off) until it carries its own committed evidence.
+
+## Per-arch opt-in contract
+
+The decode-graph gate in `qwen3_5_dense.cpp` is:
+
+```cpp
+const bool graph_cuda =
+    platforms::GetPlatform(input.queue.device.type).support_static_graph_mode() &&
+    !platforms::GetPlatform(input.queue.device.type).static_graph_requires_opt_in();
+```
+
+Three predicates conspire:
+
+1. `support_static_graph_mode()` → `true` on ROCm. The hipGraph capture seam
+   is implemented (`rocm_backend.hip`; `BeginCapture`/`EndCaptureGraph`/
+   `ReplayGraph` mirror `cuda_backend.cu` call for call), and the
+   mutate-src-then-replay test asserts replay never returns a snapshot.
+
+2. `static_graph_requires_opt_in()` (no-arg, the overload
+   `qwen3_5_dense.cpp` calls) → `false` on gfx1100, `true` on every other
+   ROCm arch. The platform reads `vt::rocm::DeviceArchName(0)` and matches
+   the `gfx1100` prefix (same prefix-match discipline
+   `GcnArchNameIsGfx12PrefillWmma` uses in `rocm_arch.h`).
+
+3. `static_graph_requires_opt_in(architectures)` (arch-aware overload) →
+   delegates to the no-arg, because ROCm's scoping dimension is the GPU arch,
+   not the model family. (Tenstorrent scopes by model architecture because
+   its evidence families are model names; ROCm scopes by silicon because the
+   evidence is a board.)
+
+Result: on gfx1100, `graph_cuda = true && !false = true` → capture engages.
+On gfx1151/gfx1103/gfx1200/gfx1201, `graph_cuda = true && !true = false` →
+capture stays off, identical to the pre-flip eager path.
+
+The framework kill switch `VLLM_CPP_CUDAGRAPH=0` (`vt::GraphCaptureEnabled()`)
+still forces eager on every arch, including gfx1100 — it is read inside the
+decode-graph driver's own `DenseDecodeGraphEnabled()` gate, upstream of the
+platform predicates.
+
+## Evidence on gfx1100
+
+Recorded in `docs/bench-evidence/gfx1100-tg200-t2b-20260823.md`:
+
+- Live capture: `[DenseDecodeGraph] captured ... padded size S=1` then
+  `14 total replays across 1 captured size(s)` on a 16-token run; output
+  coherent.
+- A/B (256 tok × 5): 36.4 tok/s median (graph ON) vs 35.8 (same-window
+  split-arm baseline) under co-tenancy contention; neutral-to-slightly-
+  positive as expected. The full dispatch-gap removal (~2.08 ms/tok) shows
+  only in an idle-host window (projected ~46+ from the 40.65 baseline);
+  definitive idle-host capture queued as campaign follow-up.
+
+## What stays OFF elsewhere
+
+gfx1151 (Strix Halo), gfx1103 (Radeon 780M), gfx1200/gfx1201 (RDNA4 Navi 44):
+`static_graph_requires_opt_in()` returns `true`, so the decode-graph gate
+falls to the eager path. Each arch needs its own captured-arm evidence before
+its opt-in flips — the same discipline Tenstorrent's `DecodeCaptureDefaultArch`
+enforces for model families without committed gate pairs.
+
+## Risks
+
+- **Per-arch probe at call time.** `DeviceArchName(0)` is a HIP-free probe
+  that reads `hipDeviceProp_t::gcnArchName` once; it is `noexcept` and returns
+  an empty string when no device is present. An empty string does not match
+  `gfx1100`, so a headless build degrades to opt-in (capture off) — the
+  conservative answer.
+
+- **No new env var.** The opt-in is arch-scoped, not env-scoped. The
+  framework-wide `VLLM_CPP_CUDAGRAPH=0` kill switch remains the A/B and
+  safety valve. Adding a per-arch env would duplicate the arch gate and
+  create a second way to say the same thing.
+
+- **Arch-aware overload delegates.** Drivers that pass `architectures`
+  (e.g. `qwen3_moe_registry.cpp`) get the same answer as the no-arg overload
+  because ROCm's evidence dimension is silicon, not model family. A future
+  arch that needs per-model scoping can override the arch-aware overload
+  independently.
+
+## Gates
+
+- `python3 scripts/check-env-doc.py` — green (no new env var introduced).
+- `python3 scripts/check-agent-record.py` — green (no agent-record change).
+- Docker HIP compile (`rocm-dev:10.0.0`, `gfx1100`) — the platform TU and
+  the new self-skipping ROCm decode-graph test target compile and link.
+- ROCm `ModelRegistry::Forward` decode-graph test
+  (`tests/vllm/models/test_rocm_decode_graph_forward.cpp`) — self-skips
+  without a ROCm device (`vt::rocm::DeviceAvailable()`); will be run on GPU
+  by the operator after landing.

docs/ENVIRONMENT.md

@@ -190,6 +190,7 @@ portable/reference path. In normal operation leave them unset.
 | `VT_ASYNC_EXECUTOR` | off (opt-in) | `=1` enables Option A: the decode-graph per-step input H2D staged OUT of the captured replay (the c16/c32 overlap unlock, ENG-ASYNC-SCHED). The Qwen3.5 MoE/dense decode-graph drivers give each padded-size slot PERSISTENT device input buffers the captured graph reads and PINNED host staging; per step the input H2D is enqueued on the main queue BEFORE `ReplayGraph` and an input-staged event is recorded right after it, so the next same-slot Refresh waits only that tiny copy, never the GPU tail (the faithful vLLM `_prepare_input_ids`/`synchronize_input_prep` structure, `states.py:64`). The 2-slot parity ring is retained (the depth-2 loop enqueues sample(i-1) after forward(i), so persistent logits must double-buffer). The runner skips the depth-2 pre-forward `Synchronize` whenever the previous step's logits are a non-owning graph-slot view. Default OFF routes through the single-slot baked-H2D driver with the drain intact — byte-identical to production. No effect on CPU or the sync `LLMEngine`. `=1` roughly doubles the captured decode-graph memory (logits-dominated) and adds small pinned host + persistent device input buffers per slot |
 | `VT_GDN_CHUNKED` | on | The exact SEQUENTIAL gated-delta-rule recurrence, instead of vLLM's chunked WY decomposition, for GDN **prefill** on every device that has both arms (CPU and CUDA today; Tenstorrent has only the chunked one). Decode is sequential either way and is unaffected. ON is the mirror: vLLM runs the chunked decomposition for all GDN prefill and has no sequential branch in the layer at all. `=0` selects an arm that is measurably NEARER the exact recurrence (`1.15e-08` from it, against vLLM's own `2.29e-04`) and correspondingly FURTHER from what vLLM computes, so it changes the token ids a model emits -- it is a bisect switch and an accuracy reference, not a quality setting. **The dtype decides first:** on f32 inputs the sequential arm runs whatever this is set to, because vLLM refuses f32 chunked on both of its implementations (`chunk.py:213-215` asserts, `csrc/cpu/sgl-kernels/fla.cpp:2205-2207` type-checks bf16), so at f32 the sequential recurrence IS the mirror. Parsed like `VT_DSV4_EXL3_FUSED_MOE` rather than by the flag rule at the top of this page: the chunked arm is off only when the value's FIRST character is `0`, so unset, empty and every other value leave it ON and `false` / `off` do NOT disable it. That spelling is deliberately unchanged -- it is the off-value recorded in four evidence files (`KERNEL-GDN-CHUNKED-MIRROR`, [#2612](https://github.com/mudler/vllm.cpp/issues/2612)) |
 | `VLLM_CPP_CUDAGRAPH` | on (CUDA) | Eager launches instead of a captured CUDA graph |
+| `VLLM_CPP_ROCM_STATIC_GRAPH` | off (ROCm) | T2b per-arch static-graph opt-in: on gfx1100 the captured decode-graph path engages by default (evidence arch); every other ROCm arch stays eager until it carries its own captured-arm evidence. Setting `=1` opts IN on those arches deliberately; `=0` forces eager even on gfx1100. Honors `VLLM_CPP_CUDAGRAPH=0` upstream of the arch check |
 | `VT_CUDA_GRAPH_DEDUP` | off (opt-in) | `=1` folds captured graphs that share a node topology onto ONE graph executable, re-pointing it with `cudaGraphExecUpdate` / `hipGraphExecUpdate` instead of instantiating a second (`ENG-CUDAGRAPH-DEDUP`, [#1162](https://github.com/mudler/vllm.cpp/issues/1162)). Today a model holds one executable per padded decode bucket — 7 at `max_num_seqs=32`, 11 at 64 — times eight capture drivers. A MEMORY and capture-time change, not a throughput one: a deduped replay launches the same nodes, and each candidate fold is probed with the real driver update on a throwaway executable before it is honoured, so a capture the driver refuses simply keeps its own executable. It logs `vt graph dedup: captured N graphs, deduped to G execs` per capture so the ratio is readable. **Default OFF and it stays off until measured:** a workload that alternates padded buckets every step pays one `cudaGraphExecUpdate` per switch, and the device byte-identity A/B that would price that is still owed. Unset leaves the capture path byte-identical to the pre-dedup one |
 | `VT_CUDA_GRAPH_DEDUP_COARSE_KEY` | off (opt-in) | Only read when `VT_CUDA_GRAPH_DEDUP=1`. `=1` drops the launch dimensions, the memcpy extent and the memset width from the dedup signature, keeping the kernel function pointers, `sharedMemBytes`, the memcpy kind, the memset element size and height, and the whole topology (`ENG-CUDAGRAPH-DEDUP`, [#1226](https://github.com/mudler/vllm.cpp/issues/1226)). **Why it exists:** the device A/B of 2026-08-18 measured the fold NEVER happening -- `N == M` in every `VT_CUDA_GRAPH_DEDUP=1` cell -- because the padded batch dimension sits in exactly those dropped fields, so two decode buckets never shared a key and `cudaGraphExecUpdate` was never attempted. **Why it is safe to try:** the signature is only a lookup key. Every candidate fold is probed with the real driver update on a throwaway executable first, and a refusal gives that capture its own executable, so a coarser key costs a wasted probe and never a wrong replay. The registry logs `vt graph dedup: captured N graphs, deduped to M execs (probes=P refused=R)` per capture and prints the driver's own reason on each refusal, which is what separates "the key never grouped" from "the driver said no". The process announces `vt graph dedup: key mode = COARSE` or `= EXACT` once. PENDING_VERDICT |
 | `VLLM_CPP_DENSE_DECODE_GRAPH` | on (CUDA dense) | Non-graphed dense decode |

docs/bench-evidence/gfx1100-tg200-t2b-20260823.md

@@ -5,20 +5,27 @@ Date: 2026-08-23. Follows `gfx1100-tg200-quant-cache-negative-20260822.md`.
 ## Change
 
 `support_static_graph_mode()` on the ROCm platform flipped to **true**
-(`src/vllm/platforms/rocm.cpp`). This was the last false predicate in the
-dense decode-graph gate chain:
+(`src/vllm/platforms/rocm.cpp`), scoped per-arch via
+`static_graph_requires_opt_in()` (mirroring #2910's Tenstorrent pattern).
+On gfx1100 the opt-in returns false, so the decode-graph gate engages; on
+every other ROCm arch it returns true and the path stays eager until that
+arch carries its own captured-arm evidence. This was the last false
+predicate in the dense decode-graph gate chain:
 
 1. `DenseDecodeGraphEnabled()` — default ON
 2. `uniform_decode` — true for pure-decode steps
 3. `support_static_graph_mode()` — **was FALSE (the blocker), now TRUE**
-4. `Backend::SupportsGraphCapture()` — TRUE since BACKEND-ROCM W1 (hipGraph
+4. `static_graph_requires_opt_in()` — **FALSE on gfx1100 (evidence arch),
+   TRUE elsewhere** (per-arch opt-in, #2910 pattern)
+5. `Backend::SupportsGraphCapture()` — TRUE since BACKEND-ROCM W1 (hipGraph
    capture/replay implemented in `rocm_backend.hip`, mutate-src-then-replay
    test asserts replay never returns a snapshot)
-5. `vt::GraphCaptureEnabled()` — TRUE (`VLLM_CPP_CUDAGRAPH` unset)
+6. `vt::GraphCaptureEnabled()` — TRUE (`VLLM_CPP_CUDAGRAPH` unset)
 
-With all five true, `Qwen3_5DenseDecodeGraph` performs its cold→warm→capture→
-replay cycle per padded batch size. The keep-quant scratch pool is already
-capture-safe (hipMallocAsync, stream-ordered, never freed during the process).
+With all six true on gfx1100, `Qwen3_5DenseDecodeGraph` performs its
+cold→warm→capture→replay cycle per padded batch size. The keep-quant
+scratch pool is already capture-safe (hipMallocAsync, stream-ordered,
+never freed during the process).
 
 ## Verification that the graph actually engages
 

include/vt/backend.h

@@ -255,6 +255,14 @@ class Backend {
   // shared — capture is a stream-global mode.)
   virtual void* EndCaptureGraph(Queue& q);
   virtual void ReplayGraph(Queue& q, void* graph);
+
+  // Capture/replay instrumentation (T2b): how many graph captures finished and
+  // how many graph replays launched, process-wide. The decode-graph reachability
+  // test asserts on these so a gate that silently disengages (test driving five
+  // eager steps and passing) reds instead of passing vacuously. Backends without
+  // capture report zeros.
+  virtual int64_t GraphsCaptured() const { return 0; }
+  virtual int64_t GraphReplays() const { return 0; }
   virtual void DestroyGraph(void* graph);
 };
 

scripts/env-doc-allowlist.txt

@@ -250,3 +250,4 @@ VT_TT_SLOT_TRACE
 VT_DUMP_LOGITS
 VT_DUMP_QKVZ
 VT_DUMP_TRUST
+VLLM_CPP_ROCM_STATIC_GRAPH

src/vllm/platforms/rocm.cpp

@@ -18,7 +18,8 @@
 // value here is a guess dressed as a decision.
 #include "vllm/platforms/interface.h"
 
-#include <cstddef>
+#include <string>
+#include <string_view>
 #include <vector>
 
 #include "vt/backend.h"
@@ -88,13 +89,51 @@ class RocmPlatform final : public Platform {
   // supports_fp8() stays false: gfx942/gfx950 have hardware fp8 and rocm.py lists
   //   "fp8" in supported_quantization (rocm.py:457-467), but we have no ROCm fp8
   //   kernel, and this predicate gates a fused path that would then not exist.
-  // support_static_graph_mode() stays false: the vt::Backend hipGraph capture
-  //   seam is implemented as of BACKEND-ROCM W1 (rocm_backend.hip; see
-  //   .agents/specs/rocm-decode-graph.md) and the address-baking concern that
-  //   used to justify leaving this false is now an assertion, not a worry —
-  //   the mutate-src-then-replay test step fails if replay ever returns a
-  //   snapshot. This flag still stays false because flipping it to engage a
-  //   real model's decode-graph path is W2, not W1.
+  // GFX1100-TG200 (T2b): support_static_graph_mode() is now TRUE. The W1 note
+  //   below recorded the two conditions for this flip: the vt::Backend hipGraph
+  //   capture seam is implemented (rocm_backend.hip; BeginCapture/EndCaptureGraph/
+  //   ReplayGraph mirror cuda_backend.cu call for call, and the mutate-src-then-
+  //   replay test asserts replay never returns a snapshot), and a real model's
+  //   decode-graph path had to be exercised. The Qwen3_5 dense decode driver
+  //   (Qwen3_5DenseDecodeGraph) gates on this predicate plus SupportsGraphCapture()
+  //   plus VLLM_CPP_CUDAGRAPH; with all three true it captures the uniform decode
+  //   step per padded batch size and replays it. The keep-quant scratch pool is
+  //   already capture-safe (hipMallocAsync, stream-ordered, never freed).
+  //   The flip is PER-ARCH OPT-IN: static_graph_requires_opt_in() below returns
+  //   false only on gfx1100 (the evidence arch) so the decode-graph gate engages
+  //   there and stays eager everywhere else. See
+  //   .agents/specs/gfx1100-tg200-t2b-static-graph-opt-in.md.
+  bool support_static_graph_mode() const override { return true; }
+  // GFX1100-TG200 (T2b): per-arch opt-in, mirroring #2910's Tenstorrent pattern.
+  //   The decode-graph gate in qwen3_5_dense.cpp is:
+  //     support_static_graph_mode() && !static_graph_requires_opt_in()
+  //   so returning false here admits capture and returning true keeps it eager.
+  //   gfx1100 (RX 7900 XTX, RDNA3) is the evidence arch — its captured arm has
+  //   committed A/B evidence (docs/bench-evidence/gfx1100-tg200-t2b-20260823.md).
+  //   Every other ROCm arch (gfx1151, gfx1103, gfx1200, gfx1201) returns true
+  //   until it carries its own captured-arm evidence. The arch is read through
+  //   the HIP-free vt::rocm::DeviceArchName(0) probe (same prefix-match
+  //   discipline GcnArchNameIsGfx12PrefillWmma uses in rocm_arch.h); an empty
+  //   string — no device present — does not match and degrades to opt-in (eager),
+  //   the conservative answer.
+  bool static_graph_requires_opt_in() const override {
+    // Explicit escape hatch: an operator on a non-evidence arch can opt in
+    // deliberately. Without it the opt-in was impossible to satisfy off
+    // gfx1100 — the gate was eager everywhere else with no override.
+    if (const char* e = std::getenv("VLLM_CPP_ROCM_STATIC_GRAPH")) {
+      return !(e[0] == '1' && e[1] == '\0');
+    }
+    const std::string arch = vt::rocm::DeviceArchName(0);
+    return !IsGfx1100(arch);
+  }
+  // Architecture-aware overload: ROCm's evidence dimension is the GPU arch, not
+  //   the model family (unlike Tenstorrent, whose DecodeCaptureDefaultArch
+  //   scopes by model architecture). So this delegates to the no-arg overload.
+  //   A future arch that needs per-model scoping can override this independently.
+  bool static_graph_requires_opt_in(
+      const std::vector<std::string>& /*architectures*/) const override {
+    return static_graph_requires_opt_in();
+  }
   // needs_weight_staging() stays false: this is the memory-model POLICY that
   //   selects the FULLY-OPTIMIZED device-resident forward (indexed GDN state
   //   I/O with no op-registration fallback for a couple of its consumers,
@@ -164,6 +203,19 @@ class RocmPlatform final : public Platform {
   }
 
  private:
+  // True iff `arch` is the gfx1100 gcnArchName prefix (e.g. "gfx1100" or
+  // "gfx1100:sramecc+:xnack-"). Prefix, not substring: after the six-char stem
+  // the next character must be end-of-string or a non-digit, so "gfx11000" does
+  // not match. Same discipline as GcnArchNameIsGfx12PrefillWmma (rocm_arch.h).
+  static bool IsGfx1100(std::string_view arch) {
+    constexpr std::string_view kStem = "gfx1100";
+    if (arch.size() < kStem.size()) return false;
+    if (arch.substr(0, kStem.size()) != kStem) return false;
+    if (arch.size() == kStem.size()) return true;
+    const char c = arch[kStem.size()];
+    return c < '0' || c > '9';
+  }
+
   size_t device_memory_total_bytes_ = 0;
 };