docs: correct Qwen3.8-Flash-Next generation claims

.agents/completed/qwen4exp-features-history-20260914.md

@@ -0,0 +1,11 @@
+# Qwen3.8-Flash-Next feature entry before 14 September 2026
+
+This superseded row is historical evidence, not the current capability record.
+The block preserves the original Markdown bytes. Its relative links use the
+original `docs/FEATURES.md` location. Read the
+[original row and its links](https://github.com/mudler/vllm.cpp/blob/cef9f8216/docs/FEATURES.md)
+for navigation.
+
+```markdown
+| `Qwen4ExpForConditionalGeneration` | GGUF (`qwen4exp`) — **LOADS, DECODES, AND SERVES ONE SEQUENCE AT A TIME ON `--device cpu`** (W5a, W5f, W5g, W5k, W5L, #2031) | **THE GGUF LOAD IS GATED, AND SO IS THE LAYER LOOP; WHAT DOES NOT EXIST IS A SECOND STEP.** A `qwen4exp` file reaches the architecture's own config builder through the GGUF dispatch, the registry resolves the class, and `load_weights` materializes the whole text tower — **on CPU **and, since KGATHER, on CUDA** ([spec](../.agents/specs/cuda-quant-gather.md)). The 51.2 G-parameter n-gram table would expand from 26.822 GiB of IQ4_NL to 95.368 GiB of bf16 on a device that cannot gather blocks, which the on-disk #1123 device-fit guard cannot see, so the load REFUSES BY NAME on such a device ahead of any tensor I/O (#2083). `DeviceQuantGatherSupported` is no longer a device list at all: it asks `OpRegistered(kEmbeddingQuant, dev)`, CPU and CUDA register that op, and METAL, VULKAN, ROCM and TENSTORRENT do not and are still refused ([#2394](https://github.com/mudler/vllm.cpp/issues/2394)). So this row's earlier clause that the gate 'is true for CPU alone' is FALSE and is replaced — every convert-time transform inverted (the `+1` fold on every norm gamma with `ssm_norm` the one exception, `ssm_a` back to `log(-x)`, and the V-head reorder on every Gated DeltaNet tensor), gated in both directions against a committed 1224-tensor manifest of the shipped `unsloth/Qwen3.8-Flash-Next-GGUF UD-IQ1_S` and value-wise against a synthetic file. `Qwen4ExpTextModel::Forward` now exists (W5f, #2336) and `ModelRegistry::Forward` reaches it: the 48-layer loop composes all four block seams and is gated END TO END against the lane-pinned transformers 5.16.0 oracle at a tiny config, max|diff| 0.00982 against a 0.03 bf16-vs-f32 bound, with seven mutations separating by 0.78 to 2.02. **ONE TOKEN NOW COMES OUT OF A PRODUCTION SEAM, and the claim is exactly that and no more** (W5g): on a model loaded by `ModelRegistry::Load` from a synthetic `qwen4exp` GGUF, `ModelRegistry::Forward` returns `[1, vocab]` f32 logits with every element finite and `vt::GreedyArgmax` samples an id from them, on CPU. It is a REACH and a SAMPLE, not a token gate — the fixture's weights are a deterministic ramp, so the id is compared against no reference, and the tower's arithmetic is gated separately by the oracle case above. W5g is what made the prefill complete at all: `Qwen4ExpPleLayout` derived the n-gram head vocabulary from a DEFAULTED `ngram_vocab_size_base` and refused when a file's stated sizes disagreed, which a `qwen4exp` GGUF cannot avoid because llama.cpp #27742's converter writes the resolved arrays and no base — so the check held for exactly one artifact in existence and refused every other file with correctly loaded weights. The stated set is now the authority for the layout, as `NgramTableRows` already treated it, and the cross-check runs only where the source stated the base. **THE ENGINE'S CACHE CHANNEL NOW REACHES THIS MODEL** (W5j, #2031, #2353). This row said the `multi_kv` channel "is refused for every model by `ModelRegistry::Forward`"; that guard is now a MODEL-DECLARED capability, `ModelFactory::consumes_multi_kv`, landed with its first consumer, and this architecture is that consumer. A step carrying all three published groups reaches the hook, which resolves every one of its five caches BY NAME through `MultiKvCacheIndex::Resolve` — including the recurrent members, which `ENG-MULTIKV-BYNAME` made addressable — and reads the QSA indexer side cache out of the engine's own group-2 pages through group 2's own gathered block table. The guard still refuses `DeepseekV4ForCausalLM`, `Glm5NextForConditionalGeneration` and every architecture that declares nothing, and clearing the bit drives that refusal red in the gate. **IT NOW DECODES, AND IT NOW SERVES** (W5k and W5L, #2031). This row said "it still decodes NO token, and the reason is now the MODEL and not the engine", and named a dtype and a residency: the recurrent group publishes the PLE conv ring at the model dtype while `RunQwen4ExpPleBlock` required f32, and publishes the n-gram token history as a device i64 state while the same block read it through a HOST pointer. W5k settled both against the RUNNING lane pin — transformers 5.16.0, `modeling_qwen4_exp.py` sha256 `77fec77d…c459`, confirmed by regenerating this row's committed forward golden byte-identically. Upstream types each cache slot from the tensor that first reaches it (`cache_utils.py:1019-1023`), so the ring carries the MODEL dtype and the history lives on the compute device: the PUBLISHER was right twice and both requirements moved to the block. `ModelRegistry::Forward` then runs a prefill and a `past_len > 0` DECODE over one set of persistent caches. **W5L drives the engine itself.** A real `GPUModelRunner` allocates all three published groups, gathers every group's block table, publishes the five-name by-name index and runs `execute_model` / `sample_tokens` for a prefill and then a decode; `LoadedEngine::FromModelDir` loads a `qwen4exp` GGUF and `generate` returns tokens; and `examples/server` answers `POST /v1/completions` on CPU. The cross-step gate is the PLE n-gram history read out of the RUNNER's own recurrent state at the slot the runner assigned — int64 token ids, which cannot saturate as this fixture's bf16 activations do. **WHAT SERVES IS EXACTLY THIS AND NO MORE: `--device cpu`, ONE SEQUENCE AT A TIME, over a GGUF.** `num_reqs > 1` is refused by name — `RunQwen4ExpQsaBlockPaged` takes a block table of one sequence — and because an EngineCore that meets that refusal dies rather than degrades, this factory sets `ModelFactory::serves_one_sequence_per_step` and `LoadedEngine::ResolveMaxNumSeqs` clamps `--max-num-seqs` to 1 and says so on stderr; concurrent clients are accepted and served in sequence. MEASURED before that clamp existed: three overlapping `/v1/completions` calls at `--max-num-seqs 4` each returned a 500 carrying this hook's own message, and the engine never served again. The quant arms are the loader's — IQ4_NL, Q5_0 and the dequantizing gather (#1989), whose CUDA arm landed with KGATHER. **ALL SIX `qwen4_exp` ops PLUS `vt::RmsNormGroup` NOW HAVE CUDA ARMS** (W6-CUDA and W6-CUDA-B, #2031) and this row's earlier sentence that "no CUDA arm exists for any `qwen4_exp` op" is false and is replaced. **THAT LEAVES THE GATHER, AND KGATHER LANDED IT**, so the sentence this row carried — that the one remaining reason is `EmbeddingKernelCuda` refusing a block-quantized table — is FALSE and is replaced. `vt::Embedding` on a CUDA queue decodes a block row across all 18 encodings `vt::cpu::BlockToFloat` decodes, measured bit-exact against the CPU arm on a GPU. **WITH BOTH LANDED, WHAT BLOCKS A CUDA FORWARD IS NEITHER OP REGISTRATION NOR THE LOADER, AND THIS IS A PREDICTION RATHER THAN A MEASUREMENT:** the expected shape is partial dispatch through the PLE, then a NAMED REFUSAL at the first QSA layer, because `qwen4_exp_qsa_block.cpp` still reads three operands on the HOST — `CheckRopeLayoutsAgree`, `IndexerRows` on the block table, and `Qwen4ExpQsaIndex` on `kv_lens` — which is owned by the QSADEV wave and NOT by this row. A second wall the synthetic fixture never reaches: `IsCudaKeepQuantSupported` still excludes IQ4_NL and Q5_0, which the released UD-IQ1_S uses, owned by [#2423](https://github.com/mudler/vllm.cpp/issues/2423). **A GPU HAS NOW PRODUCED TOKENS FOR THIS MODEL, ON ROCm, AND THE CLAIM IS LIVENESS AND NOT CORRECTNESS.** Measured 2026-09-13 on `strix:gpu0` (gfx1151, Radeon 8060S, ROCm 7.2.4) inside an `rc` lease ([spec](../.agents/specs/rocm-chunked-pinned-h2d.md)): the released `unsloth/Qwen3.8-Flash-Next-GGUF` UD-IQ1_S loads through `--device auto` and `examples/vllm-cli` returns 32 tokens with `finish_reason=length` on three of three launches -- fluent, coherent, prompt-dependent output, byte-identical across the three. Steady-state decode is **5.0-5.3 tok/s** -- nine samples across six independent process launches, 5.002 to 5.291 tok/s, 5.8% max-to-min. It is a RANGE: an earlier two-launch reading quoted 5.27-5.29 at a 0.34% spread and that precision does not reproduce. **IT IS NOT A TOKEN GATE AND IT IS NOT A PARITY CLAIM, AND THE REASON THIS ROW USED TO GIVE IS FALSE AND IS REPLACED RATHER THAN AMENDED.** It read "this architecture has no GPU oracle at all: llama.cpp aborts in `build_delta_net_chunking` before it reads a byte and no vLLM revision implements `qwen4_exp`, so nothing decoded these prompts beside us, there is no denominator". Only the STOCK `llama-cpp` pin `b10451` aborts there. The scoped [`llama-cpp-qwen4exp`](../.agents/oracles/llama-cpp-qwen4exp.md) oracle (PR #27742 at `035e22731a`, `gateable = yes`) builds with HIP for `gfx1151` and decoded this same UD-IQ1_S artifact on `strix:gpu0` at **25.877 tok/s** (median of 12 legs, 1.734% leg spread, [evidence](bench-evidence/qwen4exp-llamacpp-denominator-gfx1151-20260913.md)), and vLLM registers `Qwen4ExpForConditionalGeneration` at the ACTIVE parity pin `e126687a9a` (`registry.py:580`). **So a denominator EXISTS, and a primary oracle that could define token-exact EXISTS.** What is missing is a primary-oracle RUN: `cooperative_topk` in the QSA indexer refuses to launch on this fleet at that revision ([#2626](https://github.com/mudler/vllm.cpp/issues/2626)), every published safetensors arm exceeds the largest fleet box while vLLM cannot open the GGUF, and the decode GEMM plan is `sm_103`-gated. **The two figures are still NOT divided and no speed comparison is admissible here**, because `AGENTS.md` §Gates admits a performance result only after this arm's declared token-exact gate passes and it has none ([`ISSUE-LOCAL-01M2D6MV5RNSSM2GZVZKCZA4EG`](../.agents/issues/MODEL-MM-QWEN4-EXP/ISSUE-LOCAL-01M2D6MV5RNSSM2GZVZKCZA4EG.md)); the llama.cpp arm is also TEXT-ONLY while this row is a multimodal port. What reaches a token is the bounded pinned host-to-device ring; the same binary with `VT_ROCM_PINNED_H2D_MIB=0` stops at 29.69 GiB of device memory and produces nothing in 1200 s. **NO TOKEN HAS COME OUT OF A CUDA DEVICE FOR THIS MODEL and none is claimed** -- that half of the previous sentence is unchanged, and it is owed by the QSADEV host-operand wave and [#2423](https://github.com/mudler/vllm.cpp/issues/2423), not by the ROCm row. **NO TOKEN NUMBER AND NO SPEED NUMBER**: everything above ran on a synthetic fixture whose weights are a deterministic ramp, and the safetensors arm refuses because every published safetensors artifact exceeds every device this project owns. **THE CLAUSE 'no published `qwen4exp` checkpoint has been served' IS NOW HALF FALSE AND IS REPLACED BY A MEASUREMENT.** The released `unsloth/Qwen3.8-Flash-Next-GGUF` UD-IQ1_S (67.564 GiB, 3 shards, 1224 tensors) was driven through `examples/server` on `thor:gpu0` on 2026-08-30 (`rc` job `0f188dd1`, [evidence](bench-evidence/qwen4exp-released-checkpoint-serve-20260830.md)): **it LOADED and the server LISTENED, in 4446 s at 69.206 GiB peak RSS, keeping every one of its nine encodings quantized -- and it produced ZERO TOKENS.** `POST /v1/completions` returned 500 because the forward refused the artifact by name: `vt: qwen4_exp_gated_residual: input_mix_weight_down must be float (f32/bf16 for outputs)`. The file stores all **194** hyper-connection mix weights as Q8_0, the loader correctly keeps them quantized, and `vt::Qwen4ExpGatedResidual` accepted only float; every arm of the synthetic fixture wrote those same tensors as F32, which is why every gate on this row was green and none of them could see it. **W5p REMOVED THAT REFUSAL AT ITS SOURCE** (#2031): `mix_down`, `mix_up` and `block_inject` now accept a block-quantized `[N,K]` weight and route through `vt::MatmulBT`, which dispatches the keep-quant GEMM `kMatmulBTQuant` -- mirroring llama.cpp, which merged this architecture on 2026-08-27 (`6c84c7d5d`, first tag `b10660`), declares all six of these projections `GGML_OP_MUL_MAT` and never dequantizes one. The ELEMENTWISE operands did not move: a block-typed `hc_*_norm` gamma is still refused by name, which is llama.cpp's own split (`GGML_OP_MUL` for the norm, with an explicit f32 cast where a file-typed weight meets an elementwise multiply). The synthetic fixture grew the arm that was missing (`FixtureOpts::hc_mix_q8_0`), and `ModelRegistry::Forward` runs a prefill and a second prompt over a Q8_0-mix file; restoring the old contract reds that case with the verbatim string above, which is what makes the reach measured. **W5q RE-RAN THE RELEASED CHECKPOINT THROUGH THE REPAIRED PATH, AND THE REFUSAL IS GONE WHILE THE OUTPUT IS DEGENERATE** ([evidence](bench-evidence/qwen4exp-released-checkpoint-serve-20260831.md)): on `thor:gpu0` `--device cpu`, staged to worker-local disk, the artifact loads in **61 s** (against 4446 s from the CIFS share) at `VmHWM` 73.935 GiB, a 5-token prefill and eight decode steps run with nothing thrown, and `POST /v1/completions` returns **HTTP 200** with 8 tokens where W5n got a 500. **But every one of those tokens is id 0 — `!` in this file's own vocabulary — and the answer is BYTE-IDENTICAL for two different prompts.** So the forward is degenerate and prompt-independent on the real weights, no usable token has yet come out of a published `qwen4exp` checkpoint, and the DECODES and SERVES claims at the head of this row remain true OF THE FIXTURE. **W5s THEN GOT REAL TOKENS OUT OF IT, AND THE CAUSE WAS THE REPACK MARKER** ([evidence](bench-evidence/qwen4exp-released-checkpoint-tokens-20260831.md)): on `origin/main` `52f7ccbfc`, which carries W5r as well as W5p, the same artifact on the same box answers `" Paris. Given this fact, what is"` and `" 100°C at sea level"` — two different prompts, two different prompt-dependent completions, eight distinct token ids none of them 0. W5q's tree predated W5r, so on `thor` (aarch64 i8mm, where `vt::cpu::QuantRepackActive()` is TRUE) `dense_attn::ResidentWeight` was still dropping the repack marker and `kMatmulBTQuant` read `block_q8_0x4` buffers as flat `q8_0` on every hyper-connection mix weight; a read-only per-stage probe puts a NaN in `stream.after_layer_0` (`nan=51200`) collapsing to an all-zero `LOGITS` row (`zero=248320`), and `argmax` over a row with no maximum returns index 0. Post-W5r that stage is `nan=0` and the logit row is `min -9.89818 max 15.7873` with argmax id 11751 = the `" Paris"` token; `VT_CPU_QUANT_REPACK=0` is byte-identical to the default, which is what a correct performance transform must be. **WHAT RUNS ON A PUBLISHED CHECKPOINT IS EXACTLY THIS: the UD-IQ1_S GGUF arm, ONE SEQUENCE AT A TIME, on `--device cpu` -- and, for LIVENESS ONLY, on ROCm `--device auto` on one gfx1151 board as the paragraph above records -- and no more.** **IT IS NOT A TOKEN GATE** — no oracle decoded these prompts, the STOCK `llama-cpp` pin `b10451` aborts in `build_delta_net_chunking` before loading a byte (the scoped [`llama-cpp-qwen4exp`](../.agents/oracles/llama-cpp-qwen4exp.md) oracle does NOT, and it has since decoded this artifact on `gfx1151`), the other six published quants are unrun, and there is no speed number. The repaired route is also a per-TOKEN matvec where llama.cpp batches the projection over the whole prefill; batching it is owed and unmeasured. The shipped GGUF is TEXT-ONLY (1224 tensors, no `v.blk.*`), so the multimodal arm has no artifact to load either. **CONFIG LAYER GATED as well.** The config resolves and validates against a RUNNING transformers 5.16.0 oracle (it imports without torch, so `validate_architecture` executes): a 39-case two-direction sweep agrees on 35 and differs on 4, all four being local guards stricter than upstream, never looser. All 15 upstream `validate_architecture` rejections are implemented and tabulated against their upstream line. The forward and the KV-cache spec REFUSE BY NAME, each naming the wave that owes it. vLLM implements `qwen4_exp` at NO revision, so the algorithm oracle is transformers **5.16.0** under an accepted lane exception; `gateable = no` because nothing published fits a fleet device — `Qwen/Qwen3.8-Flash-Next` is ~360 GB bf16, ~180 GB FP8, ~128 GB NVFP4 against ~119.6 GiB usable on GB10 | none, and no speed claim is admissible from this row until a token gate exists |
+```

.agents/issues/_owed/ISSUE-LOCAL-01M2EXYPHDZ03EX01VRTB5FECQ.md

@@ -0,0 +1,19 @@
+ID: ISSUE-LOCAL-01M2EXYPHDZ03EX01VRTB5FECQ
+Title: Qwen3.8-Flash-Next feature entry contradicts current generation evidence
+Row: -
+State: OPEN
+Kind: docs
+GitHub: -
+Mirror: PENDING
+Availability: FULL
+Created: 2026-09-14
+Updated: 2026-09-14
+Closed: -
+
+## Problem
+
+The feature cell says no second step, no CUDA tokens, and no vLLM implementation despite later source and evidence. Replace attempt history with current limits and add README news for the reachable text path.
+
+## Resolution
+
+-