From 12abed189bbe38779305904512411ca1ca0124ef Mon Sep 17 00:00:00 2001 From: localai-org-maint-bot <306269227+localai-org-maint-bot@users.noreply.github.com> Date: Mon, 14 Sep 2026 03:04:47 +0000 Subject: [PATCH 1/3] spec: scope the Qwen3.8-Flash-Next docs correction The feature entry contradicts later generation evidence and repeats obsolete attempts. Scope a CPU-only correction with source checks and independent review. Tracks ISSUE-LOCAL-01M2EXYPHDZ03EX01VRTB5FECQ. FOLLOWING_AGENTS_PROTOCOL Following-Agents-Protocol: true AI-Assisted: true Assisted-by: Codex:GPT-6 [Codex] --- .../ISSUE-LOCAL-01M2EXYPHDZ03EX01VRTB5FECQ.md | 19 ++++++ .agents/specs/qwen4-exp-public-doc-refresh.md | 65 +++++++++++++++++++ 2 files changed, 84 insertions(+) create mode 100644 .agents/issues/_owed/ISSUE-LOCAL-01M2EXYPHDZ03EX01VRTB5FECQ.md create mode 100644 .agents/specs/qwen4-exp-public-doc-refresh.md diff --git a/.agents/issues/_owed/ISSUE-LOCAL-01M2EXYPHDZ03EX01VRTB5FECQ.md b/.agents/issues/_owed/ISSUE-LOCAL-01M2EXYPHDZ03EX01VRTB5FECQ.md new file mode 100644 index 000000000..86590ec43 --- /dev/null +++ b/.agents/issues/_owed/ISSUE-LOCAL-01M2EXYPHDZ03EX01VRTB5FECQ.md @@ -0,0 +1,19 @@ +ID: ISSUE-LOCAL-01M2EXYPHDZ03EX01VRTB5FECQ +Title: Qwen3.8-Flash-Next feature entry contradicts current generation evidence +Row: - +State: OPEN +Kind: docs +GitHub: - +Mirror: PENDING +Availability: FULL +Created: 2026-09-14 +Updated: 2026-09-14 +Closed: - + +## Problem + +The feature cell says no second step, no CUDA tokens, and no vLLM implementation despite later source and evidence. Replace attempt history with current limits and add README news for the reachable text path. + +## Resolution + +- diff --git a/.agents/specs/qwen4-exp-public-doc-refresh.md b/.agents/specs/qwen4-exp-public-doc-refresh.md new file mode 100644 index 000000000..69da6b3af --- /dev/null +++ b/.agents/specs/qwen4-exp-public-doc-refresh.md @@ -0,0 +1,65 @@ +# Qwen3.8-Flash-Next public documentation refresh + +## Now + +Documentation audit at base `cef9f8216`, 14 September 2026. No model lifecycle, +runtime behavior, or benchmark acceptance changes. Implementation and independent +review are pending. One pull request carries the spec and documentation commits. + +## Scope + +Replace the contradictory `Qwen4ExpForConditionalGeneration` entry in +`docs/FEATURES.md` with a concise account of current behavior. Add one README +news item for real-checkpoint text generation on CPU and ROCm. Preserve the +superseded feature cell verbatim in an era-stamped file under `.agents/completed/`. +Do not edit the model matrix, kernel code, benchmark numbers, or unrelated docs. +The existing EXL3 and HTTP multimodal documentation pull requests own those topics. + +## Source and evidence inventory + +| Surface | Authority | Required disposition | +|---|---|---| +| Production decode | `src/vllm/model_executor/models/qwen4_exp.cpp`, `qwen4_exp_forward.cpp`, and the model registry | Verify actual registration, decode routing, and one-sequence limit | +| Backend dispatch | `src/vt/rocm/rocm_qwen4_exp.hip`, quantized embedding registration and loader guards | Distinguish reachable code from measured real-checkpoint output | +| CPU checkpoint | `docs/bench-evidence/qwen4exp-released-checkpoint-tokens-20260831.md` | Prompt-dependent generation, no oracle token gate | +| ROCm checkpoint | `docs/bench-evidence/qwen4exp-rocm-hcnorm-gfx1151-20260913.md` | Generation on gfx1151, correctness and competitive speed still ungated | +| CUDA and artifacts | `docs/USAGE.md`, retained CUDA token evidence, owning model spec | No obsolete claim that CUDA emits no tokens; preserve actual unresolved token disagreement | +| Oracle | `.agents/oracles/vllm.md`, `llama-cpp-qwen4exp.md`, owning model spec | vLLM registration exists; no claim of a completed primary-oracle run | + +Read exact paths and line anchors before writing. The public feature entry must +name the tested UD-IQ1_S GGUF, single-sequence serving, text-only artifact, and +unresolved correctness. Link evidence instead of repeating attempt history. +Do not publish an engine ratio or promote a liveness measurement to parity. + +## Design and risks + +Keep the existing feature-table columns. Use short sentences and descriptive +links. README news links to the feature entry or its evidence. Do not introduce +new commands: existing usage recipes retain authority. Archived relative links +must resolve from the archive, without changing the quoted source cell itself. + +The main risk is mistaking a stale sentence for current behavior. Reconcile +conflicts against source and later committed evidence. Runtime verification is +unavailable on this CPU-only host; no new runtime claim is permitted. + +## Tests and gates + +This is a documentation correction, with no upstream test port or runtime +benchmark applicable. Capture the contradictory before text as the failing +baseline. Run existing `check-readme-structure.py`, `check-supported-models.py`, +`check-site.py`, `check-agent-record.py`, and their relevant mutation suites. +Run full preflight and distinguish unchanged baseline failures from regressions. +An independent reviewer checks every new claim against source and evidence, +checks preserved archive bytes, and mutates links in a scratch copy to verify +the link gate. The operator reruns the focused checks at the reviewed SHA. + +## Completion and stop conditions + +Done when scoped documentation is source-checked, focused checks pass, a fresh +review passes, and the fork pull request is open. No upstream merge is authorized. +Stop for a claim requiring a new GPU run; describe the gap without filling it. + +## Owed + +`ISSUE-LOCAL-01M2EXYPHDZ03EX01VRTB5FECQ` owns this documentation correction. +The model's existing runtime issues remain with its owning spec. From f450f76261fa2bf82ab7df6d9b734e884971dc5b Mon Sep 17 00:00:00 2001 From: localai-org-maint-bot <306269227+localai-org-maint-bot@users.noreply.github.com> Date: Mon, 14 Sep 2026 03:09:51 +0000 Subject: [PATCH 2/3] docs: correct Qwen3.8-Flash-Next generation claims The feature row contradicted retained CPU and CUDA generation evidence. Replace its attempt history with current behavior and unresolved gates. Announce real-checkpoint CPU and ROCm generation in the README. Preserve the original row verbatim for historical reference. Issue: ISSUE-LOCAL-01M2EXYPHDZ03EX01VRTB5FECQ FOLLOWING_AGENTS_PROTOCOL Following-Agents-Protocol: true AI-Assisted: true Assisted-by: Codex:GPT-6 [Codex] --- .../qwen4exp-features-history-20260914.md | 11 ++++ .agents/specs/qwen4-exp-public-doc-refresh.md | 65 ++++++++++++++++++- README.md | 3 + docs/FEATURES.md | 2 +- 4 files changed, 78 insertions(+), 3 deletions(-) create mode 100644 .agents/completed/qwen4exp-features-history-20260914.md diff --git a/.agents/completed/qwen4exp-features-history-20260914.md b/.agents/completed/qwen4exp-features-history-20260914.md new file mode 100644 index 000000000..2a134ed4d --- /dev/null +++ b/.agents/completed/qwen4exp-features-history-20260914.md @@ -0,0 +1,11 @@ +# Qwen3.8-Flash-Next feature entry before 14 September 2026 + +This superseded row is historical evidence, not the current capability record. +The block preserves the original Markdown bytes. Its relative links use the +original `docs/FEATURES.md` location. Read the +[original row and its links](https://github.com/mudler/vllm.cpp/blob/cef9f8216/docs/FEATURES.md) +for navigation. + +```markdown +| `Qwen4ExpForConditionalGeneration` | GGUF (`qwen4exp`) — **LOADS, DECODES, AND SERVES ONE SEQUENCE AT A TIME ON `--device cpu`** (W5a, W5f, W5g, W5k, W5L, #2031) | **THE GGUF LOAD IS GATED, AND SO IS THE LAYER LOOP; WHAT DOES NOT EXIST IS A SECOND STEP.** A `qwen4exp` file reaches the architecture's own config builder through the GGUF dispatch, the registry resolves the class, and `load_weights` materializes the whole text tower — **on CPU **and, since KGATHER, on CUDA** ([spec](../.agents/specs/cuda-quant-gather.md)). The 51.2 G-parameter n-gram table would expand from 26.822 GiB of IQ4_NL to 95.368 GiB of bf16 on a device that cannot gather blocks, which the on-disk #1123 device-fit guard cannot see, so the load REFUSES BY NAME on such a device ahead of any tensor I/O (#2083). `DeviceQuantGatherSupported` is no longer a device list at all: it asks `OpRegistered(kEmbeddingQuant, dev)`, CPU and CUDA register that op, and METAL, VULKAN, ROCM and TENSTORRENT do not and are still refused ([#2394](https://github.com/mudler/vllm.cpp/issues/2394)). So this row's earlier clause that the gate 'is true for CPU alone' is FALSE and is replaced — every convert-time transform inverted (the `+1` fold on every norm gamma with `ssm_norm` the one exception, `ssm_a` back to `log(-x)`, and the V-head reorder on every Gated DeltaNet tensor), gated in both directions against a committed 1224-tensor manifest of the shipped `unsloth/Qwen3.8-Flash-Next-GGUF UD-IQ1_S` and value-wise against a synthetic file. `Qwen4ExpTextModel::Forward` now exists (W5f, #2336) and `ModelRegistry::Forward` reaches it: the 48-layer loop composes all four block seams and is gated END TO END against the lane-pinned transformers 5.16.0 oracle at a tiny config, max|diff| 0.00982 against a 0.03 bf16-vs-f32 bound, with seven mutations separating by 0.78 to 2.02. **ONE TOKEN NOW COMES OUT OF A PRODUCTION SEAM, and the claim is exactly that and no more** (W5g): on a model loaded by `ModelRegistry::Load` from a synthetic `qwen4exp` GGUF, `ModelRegistry::Forward` returns `[1, vocab]` f32 logits with every element finite and `vt::GreedyArgmax` samples an id from them, on CPU. It is a REACH and a SAMPLE, not a token gate — the fixture's weights are a deterministic ramp, so the id is compared against no reference, and the tower's arithmetic is gated separately by the oracle case above. W5g is what made the prefill complete at all: `Qwen4ExpPleLayout` derived the n-gram head vocabulary from a DEFAULTED `ngram_vocab_size_base` and refused when a file's stated sizes disagreed, which a `qwen4exp` GGUF cannot avoid because llama.cpp #27742's converter writes the resolved arrays and no base — so the check held for exactly one artifact in existence and refused every other file with correctly loaded weights. The stated set is now the authority for the layout, as `NgramTableRows` already treated it, and the cross-check runs only where the source stated the base. **THE ENGINE'S CACHE CHANNEL NOW REACHES THIS MODEL** (W5j, #2031, #2353). This row said the `multi_kv` channel "is refused for every model by `ModelRegistry::Forward`"; that guard is now a MODEL-DECLARED capability, `ModelFactory::consumes_multi_kv`, landed with its first consumer, and this architecture is that consumer. A step carrying all three published groups reaches the hook, which resolves every one of its five caches BY NAME through `MultiKvCacheIndex::Resolve` — including the recurrent members, which `ENG-MULTIKV-BYNAME` made addressable — and reads the QSA indexer side cache out of the engine's own group-2 pages through group 2's own gathered block table. The guard still refuses `DeepseekV4ForCausalLM`, `Glm5NextForConditionalGeneration` and every architecture that declares nothing, and clearing the bit drives that refusal red in the gate. **IT NOW DECODES, AND IT NOW SERVES** (W5k and W5L, #2031). This row said "it still decodes NO token, and the reason is now the MODEL and not the engine", and named a dtype and a residency: the recurrent group publishes the PLE conv ring at the model dtype while `RunQwen4ExpPleBlock` required f32, and publishes the n-gram token history as a device i64 state while the same block read it through a HOST pointer. W5k settled both against the RUNNING lane pin — transformers 5.16.0, `modeling_qwen4_exp.py` sha256 `77fec77d…c459`, confirmed by regenerating this row's committed forward golden byte-identically. Upstream types each cache slot from the tensor that first reaches it (`cache_utils.py:1019-1023`), so the ring carries the MODEL dtype and the history lives on the compute device: the PUBLISHER was right twice and both requirements moved to the block. `ModelRegistry::Forward` then runs a prefill and a `past_len > 0` DECODE over one set of persistent caches. **W5L drives the engine itself.** A real `GPUModelRunner` allocates all three published groups, gathers every group's block table, publishes the five-name by-name index and runs `execute_model` / `sample_tokens` for a prefill and then a decode; `LoadedEngine::FromModelDir` loads a `qwen4exp` GGUF and `generate` returns tokens; and `examples/server` answers `POST /v1/completions` on CPU. The cross-step gate is the PLE n-gram history read out of the RUNNER's own recurrent state at the slot the runner assigned — int64 token ids, which cannot saturate as this fixture's bf16 activations do. **WHAT SERVES IS EXACTLY THIS AND NO MORE: `--device cpu`, ONE SEQUENCE AT A TIME, over a GGUF.** `num_reqs > 1` is refused by name — `RunQwen4ExpQsaBlockPaged` takes a block table of one sequence — and because an EngineCore that meets that refusal dies rather than degrades, this factory sets `ModelFactory::serves_one_sequence_per_step` and `LoadedEngine::ResolveMaxNumSeqs` clamps `--max-num-seqs` to 1 and says so on stderr; concurrent clients are accepted and served in sequence. MEASURED before that clamp existed: three overlapping `/v1/completions` calls at `--max-num-seqs 4` each returned a 500 carrying this hook's own message, and the engine never served again. The quant arms are the loader's — IQ4_NL, Q5_0 and the dequantizing gather (#1989), whose CUDA arm landed with KGATHER. **ALL SIX `qwen4_exp` ops PLUS `vt::RmsNormGroup` NOW HAVE CUDA ARMS** (W6-CUDA and W6-CUDA-B, #2031) and this row's earlier sentence that "no CUDA arm exists for any `qwen4_exp` op" is false and is replaced. **THAT LEAVES THE GATHER, AND KGATHER LANDED IT**, so the sentence this row carried — that the one remaining reason is `EmbeddingKernelCuda` refusing a block-quantized table — is FALSE and is replaced. `vt::Embedding` on a CUDA queue decodes a block row across all 18 encodings `vt::cpu::BlockToFloat` decodes, measured bit-exact against the CPU arm on a GPU. **WITH BOTH LANDED, WHAT BLOCKS A CUDA FORWARD IS NEITHER OP REGISTRATION NOR THE LOADER, AND THIS IS A PREDICTION RATHER THAN A MEASUREMENT:** the expected shape is partial dispatch through the PLE, then a NAMED REFUSAL at the first QSA layer, because `qwen4_exp_qsa_block.cpp` still reads three operands on the HOST — `CheckRopeLayoutsAgree`, `IndexerRows` on the block table, and `Qwen4ExpQsaIndex` on `kv_lens` — which is owned by the QSADEV wave and NOT by this row. A second wall the synthetic fixture never reaches: `IsCudaKeepQuantSupported` still excludes IQ4_NL and Q5_0, which the released UD-IQ1_S uses, owned by [#2423](https://github.com/mudler/vllm.cpp/issues/2423). **A GPU HAS NOW PRODUCED TOKENS FOR THIS MODEL, ON ROCm, AND THE CLAIM IS LIVENESS AND NOT CORRECTNESS.** Measured 2026-09-13 on `strix:gpu0` (gfx1151, Radeon 8060S, ROCm 7.2.4) inside an `rc` lease ([spec](../.agents/specs/rocm-chunked-pinned-h2d.md)): the released `unsloth/Qwen3.8-Flash-Next-GGUF` UD-IQ1_S loads through `--device auto` and `examples/vllm-cli` returns 32 tokens with `finish_reason=length` on three of three launches -- fluent, coherent, prompt-dependent output, byte-identical across the three. Steady-state decode is **5.0-5.3 tok/s** -- nine samples across six independent process launches, 5.002 to 5.291 tok/s, 5.8% max-to-min. It is a RANGE: an earlier two-launch reading quoted 5.27-5.29 at a 0.34% spread and that precision does not reproduce. **IT IS NOT A TOKEN GATE AND IT IS NOT A PARITY CLAIM, AND THE REASON THIS ROW USED TO GIVE IS FALSE AND IS REPLACED RATHER THAN AMENDED.** It read "this architecture has no GPU oracle at all: llama.cpp aborts in `build_delta_net_chunking` before it reads a byte and no vLLM revision implements `qwen4_exp`, so nothing decoded these prompts beside us, there is no denominator". Only the STOCK `llama-cpp` pin `b10451` aborts there. The scoped [`llama-cpp-qwen4exp`](../.agents/oracles/llama-cpp-qwen4exp.md) oracle (PR #27742 at `035e22731a`, `gateable = yes`) builds with HIP for `gfx1151` and decoded this same UD-IQ1_S artifact on `strix:gpu0` at **25.877 tok/s** (median of 12 legs, 1.734% leg spread, [evidence](bench-evidence/qwen4exp-llamacpp-denominator-gfx1151-20260913.md)), and vLLM registers `Qwen4ExpForConditionalGeneration` at the ACTIVE parity pin `e126687a9a` (`registry.py:580`). **So a denominator EXISTS, and a primary oracle that could define token-exact EXISTS.** What is missing is a primary-oracle RUN: `cooperative_topk` in the QSA indexer refuses to launch on this fleet at that revision ([#2626](https://github.com/mudler/vllm.cpp/issues/2626)), every published safetensors arm exceeds the largest fleet box while vLLM cannot open the GGUF, and the decode GEMM plan is `sm_103`-gated. **The two figures are still NOT divided and no speed comparison is admissible here**, because `AGENTS.md` §Gates admits a performance result only after this arm's declared token-exact gate passes and it has none ([`ISSUE-LOCAL-01M2D6MV5RNSSM2GZVZKCZA4EG`](../.agents/issues/MODEL-MM-QWEN4-EXP/ISSUE-LOCAL-01M2D6MV5RNSSM2GZVZKCZA4EG.md)); the llama.cpp arm is also TEXT-ONLY while this row is a multimodal port. What reaches a token is the bounded pinned host-to-device ring; the same binary with `VT_ROCM_PINNED_H2D_MIB=0` stops at 29.69 GiB of device memory and produces nothing in 1200 s. **NO TOKEN HAS COME OUT OF A CUDA DEVICE FOR THIS MODEL and none is claimed** -- that half of the previous sentence is unchanged, and it is owed by the QSADEV host-operand wave and [#2423](https://github.com/mudler/vllm.cpp/issues/2423), not by the ROCm row. **NO TOKEN NUMBER AND NO SPEED NUMBER**: everything above ran on a synthetic fixture whose weights are a deterministic ramp, and the safetensors arm refuses because every published safetensors artifact exceeds every device this project owns. **THE CLAUSE 'no published `qwen4exp` checkpoint has been served' IS NOW HALF FALSE AND IS REPLACED BY A MEASUREMENT.** The released `unsloth/Qwen3.8-Flash-Next-GGUF` UD-IQ1_S (67.564 GiB, 3 shards, 1224 tensors) was driven through `examples/server` on `thor:gpu0` on 2026-08-30 (`rc` job `0f188dd1`, [evidence](bench-evidence/qwen4exp-released-checkpoint-serve-20260830.md)): **it LOADED and the server LISTENED, in 4446 s at 69.206 GiB peak RSS, keeping every one of its nine encodings quantized -- and it produced ZERO TOKENS.** `POST /v1/completions` returned 500 because the forward refused the artifact by name: `vt: qwen4_exp_gated_residual: input_mix_weight_down must be float (f32/bf16 for outputs)`. The file stores all **194** hyper-connection mix weights as Q8_0, the loader correctly keeps them quantized, and `vt::Qwen4ExpGatedResidual` accepted only float; every arm of the synthetic fixture wrote those same tensors as F32, which is why every gate on this row was green and none of them could see it. **W5p REMOVED THAT REFUSAL AT ITS SOURCE** (#2031): `mix_down`, `mix_up` and `block_inject` now accept a block-quantized `[N,K]` weight and route through `vt::MatmulBT`, which dispatches the keep-quant GEMM `kMatmulBTQuant` -- mirroring llama.cpp, which merged this architecture on 2026-08-27 (`6c84c7d5d`, first tag `b10660`), declares all six of these projections `GGML_OP_MUL_MAT` and never dequantizes one. The ELEMENTWISE operands did not move: a block-typed `hc_*_norm` gamma is still refused by name, which is llama.cpp's own split (`GGML_OP_MUL` for the norm, with an explicit f32 cast where a file-typed weight meets an elementwise multiply). The synthetic fixture grew the arm that was missing (`FixtureOpts::hc_mix_q8_0`), and `ModelRegistry::Forward` runs a prefill and a second prompt over a Q8_0-mix file; restoring the old contract reds that case with the verbatim string above, which is what makes the reach measured. **W5q RE-RAN THE RELEASED CHECKPOINT THROUGH THE REPAIRED PATH, AND THE REFUSAL IS GONE WHILE THE OUTPUT IS DEGENERATE** ([evidence](bench-evidence/qwen4exp-released-checkpoint-serve-20260831.md)): on `thor:gpu0` `--device cpu`, staged to worker-local disk, the artifact loads in **61 s** (against 4446 s from the CIFS share) at `VmHWM` 73.935 GiB, a 5-token prefill and eight decode steps run with nothing thrown, and `POST /v1/completions` returns **HTTP 200** with 8 tokens where W5n got a 500. **But every one of those tokens is id 0 — `!` in this file's own vocabulary — and the answer is BYTE-IDENTICAL for two different prompts.** So the forward is degenerate and prompt-independent on the real weights, no usable token has yet come out of a published `qwen4exp` checkpoint, and the DECODES and SERVES claims at the head of this row remain true OF THE FIXTURE. **W5s THEN GOT REAL TOKENS OUT OF IT, AND THE CAUSE WAS THE REPACK MARKER** ([evidence](bench-evidence/qwen4exp-released-checkpoint-tokens-20260831.md)): on `origin/main` `52f7ccbfc`, which carries W5r as well as W5p, the same artifact on the same box answers `" Paris. Given this fact, what is"` and `" 100°C at sea level"` — two different prompts, two different prompt-dependent completions, eight distinct token ids none of them 0. W5q's tree predated W5r, so on `thor` (aarch64 i8mm, where `vt::cpu::QuantRepackActive()` is TRUE) `dense_attn::ResidentWeight` was still dropping the repack marker and `kMatmulBTQuant` read `block_q8_0x4` buffers as flat `q8_0` on every hyper-connection mix weight; a read-only per-stage probe puts a NaN in `stream.after_layer_0` (`nan=51200`) collapsing to an all-zero `LOGITS` row (`zero=248320`), and `argmax` over a row with no maximum returns index 0. Post-W5r that stage is `nan=0` and the logit row is `min -9.89818 max 15.7873` with argmax id 11751 = the `" Paris"` token; `VT_CPU_QUANT_REPACK=0` is byte-identical to the default, which is what a correct performance transform must be. **WHAT RUNS ON A PUBLISHED CHECKPOINT IS EXACTLY THIS: the UD-IQ1_S GGUF arm, ONE SEQUENCE AT A TIME, on `--device cpu` -- and, for LIVENESS ONLY, on ROCm `--device auto` on one gfx1151 board as the paragraph above records -- and no more.** **IT IS NOT A TOKEN GATE** — no oracle decoded these prompts, the STOCK `llama-cpp` pin `b10451` aborts in `build_delta_net_chunking` before loading a byte (the scoped [`llama-cpp-qwen4exp`](../.agents/oracles/llama-cpp-qwen4exp.md) oracle does NOT, and it has since decoded this artifact on `gfx1151`), the other six published quants are unrun, and there is no speed number. The repaired route is also a per-TOKEN matvec where llama.cpp batches the projection over the whole prefill; batching it is owed and unmeasured. The shipped GGUF is TEXT-ONLY (1224 tensors, no `v.blk.*`), so the multimodal arm has no artifact to load either. **CONFIG LAYER GATED as well.** The config resolves and validates against a RUNNING transformers 5.16.0 oracle (it imports without torch, so `validate_architecture` executes): a 39-case two-direction sweep agrees on 35 and differs on 4, all four being local guards stricter than upstream, never looser. All 15 upstream `validate_architecture` rejections are implemented and tabulated against their upstream line. The forward and the KV-cache spec REFUSE BY NAME, each naming the wave that owes it. vLLM implements `qwen4_exp` at NO revision, so the algorithm oracle is transformers **5.16.0** under an accepted lane exception; `gateable = no` because nothing published fits a fleet device — `Qwen/Qwen3.8-Flash-Next` is ~360 GB bf16, ~180 GB FP8, ~128 GB NVFP4 against ~119.6 GiB usable on GB10 | none, and no speed claim is admissible from this row until a token gate exists | +``` diff --git a/.agents/specs/qwen4-exp-public-doc-refresh.md b/.agents/specs/qwen4-exp-public-doc-refresh.md index 69da6b3af..9b56c41d1 100644 --- a/.agents/specs/qwen4-exp-public-doc-refresh.md +++ b/.agents/specs/qwen4-exp-public-doc-refresh.md @@ -3,8 +3,9 @@ ## Now Documentation audit at base `cef9f8216`, 14 September 2026. No model lifecycle, -runtime behavior, or benchmark acceptance changes. Implementation and independent -review are pending. One pull request carries the spec and documentation commits. +runtime behavior, or benchmark acceptance changes. Implementation is complete. +Independent review and the fork pull request are pending. One pull request +carries the spec and documentation commits. ## Scope @@ -63,3 +64,63 @@ Stop for a claim requiring a new GPU run; describe the gap without filling it. `ISSUE-LOCAL-01M2EXYPHDZ03EX01VRTB5FECQ` owns this documentation correction. The model's existing runtime issues remain with its owning spec. + +## Outcome + +The feature row now describes the released text-only UD-IQ1_S GGUF and its +single-sequence limit. README news announces CPU and ROCm generation without +claiming oracle correctness or competitive performance. CUDA generates tokens, +but its disagreement with CPU remains unresolved. The original feature row is +preserved verbatim in `../completed/qwen4exp-features-history-20260914.md`. + +Source checks at implementation base `12abed189`: + +- `src/vllm/model_executor/models/qwen4_exp_registry.cpp:365` refuses multiple + requests. Lines 1322 and 1327 declare the limit and register the model. +- `src/vt/rocm/rocm_ops.hip:207` registers quantized embedding on ROCm. +- `docs/bench-evidence/qwen4exp-released-checkpoint-tokens-20260831.md` records + prompt-dependent CPU output and explicitly excludes an oracle token gate. +- `docs/bench-evidence/qwen4exp-rocm-hcnorm-gfx1151-20260913.md` records generation + on gfx1151 and the missing correctness gate. +- `docs/bench-evidence/qwen4exp-cuda-decode-identifiers-20260902.md` records CUDA + output and token disagreement. `docs/USAGE.md:1045` retains the unresolved + disagreement and the artifact's text-only scope. +- `.agents/oracles/vllm.md:68` records the blocked model run. + `.agents/oracles/llama-cpp-qwen4exp.md:198` records the scoped oracle's run. + +## Verification evidence + +The captured before row contains both “NO TOKEN HAS COME OUT OF A CUDA DEVICE” +and “A GPU HAS NOW PRODUCED TOKENS”. Its claim that no usable checkpoint token +exists contradicts its later CPU result. Source and retained runs falsify those +claims. A byte comparison against `git show 12abed189:docs/FEATURES.md` confirms +that the archive preserves the complete original row. + +Commands use the temporary Python and Bash tools selected by the operator: +`PATH=/tmp/vllm-doc-tools/root/usr/bin:/tmp/vllm-doc-tools/root/bin:$PATH` and +`LD_LIBRARY_PATH=/tmp/vllm-doc-tools/root/usr/lib`. + +| Command | Result | +|---|---| +| `python3 scripts/check-readme-structure.py` | PASS, exit 0 | +| `python3 scripts/check-supported-models.py` | PASS, exit 0 | +| `python3 scripts/check-site.py` | PASS, exit 0 | +| `python3 scripts/check-agent-record.py` | PASS, exit 0 | +| `python3 -m unittest discover -s tests/scripts -p 'test_check_readme_structure.py'` | PASS, 19 cases | +| `python3 -m unittest discover -s tests/scripts -p 'test_check_supported_models.py'` | PASS, 11 cases | +| `python3 -m unittest discover -s tests/scripts -p 'test_check_site.py'` | PASS, 7 cases after the operator installed Hugo. The initial missing-Hugo error also occurred on the untouched base | +| `python3 /tmp/verify-qwen4exp-doc-links.py "$PWD"` | PASS, all 7 changed links resolve, including both heading fragments | +| `git diff --check` | PASS, exit 0 | + +The temporary link verifier reads the new README item and feature row. It checks +each relative path and compares each fragment with the target's Markdown +headings. In a scratch directory, replacing the README target with +`docs/MISSING.md#registered-architectures` fails with exit 1. Replacing its +fragment with `#missing-anchor` also fails with exit 1. Restoring the original +README passes. The source tree stays unchanged throughout both mutations. + +Full `bash scripts/agent-preflight.sh` started before edits. Its log is +`/tmp/vllm-doc-preflight-impl-base.log`. At handoff, the full sweep is still +running. The operator owns completion and comparison with the untouched-base +log `/tmp/vllm-doc-preflight-base.log`. No GPU, runtime, or new benchmark gate +applies to this documentation-only correction. diff --git a/README.md b/README.md index 3f80e29e8..8cf428133 100644 --- a/README.md +++ b/README.md @@ -37,6 +37,9 @@ ## News +- **2026-09** **Qwen3.8-Flash-Next generates text on CPU and ROCm.** The released UD-IQ1_S GGUF + runs one sequence at a time, with ROCm generation measured on gfx1151. Oracle token correctness + and competitive performance remain unverified. See the [model details](docs/FEATURES.md#registered-architectures). - **2026-09** **C ABI 26 exposes more engine controls.** Applications can select the KV cache dtype, read speculative acceptance counters, and disable the model-level sliding window. See the [C API reference](docs/reference/c-api.md#recent-abi-additions) for defaults and limits. diff --git a/docs/FEATURES.md b/docs/FEATURES.md index 18cade43a..ce6d8e8a8 100644 --- a/docs/FEATURES.md +++ b/docs/FEATURES.md @@ -160,7 +160,7 @@ speed-pending, which [BENCHMARKS.md](BENCHMARKS.md) tracks. |---|---|---|---| | `Qwen3_5ForConditionalGeneration` | Qwen3.6-27B NVFP4 (`unsloth` @`890bdef7`, `nvidia` @`0893e160`); Qwen3.5-4B BF16; **Qwen3.8-27B BF16** @`1d4bf0f2` | 27B strict 235/235 text + 32/32 image/video; 4B cached 3/3; Qwen3.8-27B 4/7 strict, 3 exact fp32 ties in band (#915) | `unsloth` 27B at/above vLLM, ModelOpt 0.85x; 4B 1.021x; 3.8-27B c4 **0.963x**, c1/c8 absolutes (#915). Loads BF16/per-tensor FP8/NVFP4 (CT+ModelOpt); `modelopt_mixed` FP8 tower NATIVE (#164), GDN qkvz merged. CUDA/CPU | | `Qwen3_5MoeForConditionalGeneration` | Qwen3.6-35B-A3B (NVFP4 text; published BF16 text + vision tower) | NVFP4 strict 315/315 vs vLLM 0.25.0; published BF16 6/7 prompts strict 16/16 vs the pin, 7th an exact tie (#910). Image/video IMPLEMENTED, NOT GATED (#891): the tower loads and runs, mm gate OWED | gate model: 0.93x to 1.03x grid; NO BF16 or mm speed claim | -| `Qwen4ExpForConditionalGeneration` | GGUF (`qwen4exp`) — **LOADS, DECODES, AND SERVES ONE SEQUENCE AT A TIME ON `--device cpu`** (W5a, W5f, W5g, W5k, W5L, #2031) | **THE GGUF LOAD IS GATED, AND SO IS THE LAYER LOOP; WHAT DOES NOT EXIST IS A SECOND STEP.** A `qwen4exp` file reaches the architecture's own config builder through the GGUF dispatch, the registry resolves the class, and `load_weights` materializes the whole text tower — **on CPU **and, since KGATHER, on CUDA** ([spec](../.agents/specs/cuda-quant-gather.md)). The 51.2 G-parameter n-gram table would expand from 26.822 GiB of IQ4_NL to 95.368 GiB of bf16 on a device that cannot gather blocks, which the on-disk #1123 device-fit guard cannot see, so the load REFUSES BY NAME on such a device ahead of any tensor I/O (#2083). `DeviceQuantGatherSupported` is no longer a device list at all: it asks `OpRegistered(kEmbeddingQuant, dev)`, CPU and CUDA register that op, and METAL, VULKAN, ROCM and TENSTORRENT do not and are still refused ([#2394](https://github.com/mudler/vllm.cpp/issues/2394)). So this row's earlier clause that the gate 'is true for CPU alone' is FALSE and is replaced — every convert-time transform inverted (the `+1` fold on every norm gamma with `ssm_norm` the one exception, `ssm_a` back to `log(-x)`, and the V-head reorder on every Gated DeltaNet tensor), gated in both directions against a committed 1224-tensor manifest of the shipped `unsloth/Qwen3.8-Flash-Next-GGUF UD-IQ1_S` and value-wise against a synthetic file. `Qwen4ExpTextModel::Forward` now exists (W5f, #2336) and `ModelRegistry::Forward` reaches it: the 48-layer loop composes all four block seams and is gated END TO END against the lane-pinned transformers 5.16.0 oracle at a tiny config, max|diff| 0.00982 against a 0.03 bf16-vs-f32 bound, with seven mutations separating by 0.78 to 2.02. **ONE TOKEN NOW COMES OUT OF A PRODUCTION SEAM, and the claim is exactly that and no more** (W5g): on a model loaded by `ModelRegistry::Load` from a synthetic `qwen4exp` GGUF, `ModelRegistry::Forward` returns `[1, vocab]` f32 logits with every element finite and `vt::GreedyArgmax` samples an id from them, on CPU. It is a REACH and a SAMPLE, not a token gate — the fixture's weights are a deterministic ramp, so the id is compared against no reference, and the tower's arithmetic is gated separately by the oracle case above. W5g is what made the prefill complete at all: `Qwen4ExpPleLayout` derived the n-gram head vocabulary from a DEFAULTED `ngram_vocab_size_base` and refused when a file's stated sizes disagreed, which a `qwen4exp` GGUF cannot avoid because llama.cpp #27742's converter writes the resolved arrays and no base — so the check held for exactly one artifact in existence and refused every other file with correctly loaded weights. The stated set is now the authority for the layout, as `NgramTableRows` already treated it, and the cross-check runs only where the source stated the base. **THE ENGINE'S CACHE CHANNEL NOW REACHES THIS MODEL** (W5j, #2031, #2353). This row said the `multi_kv` channel "is refused for every model by `ModelRegistry::Forward`"; that guard is now a MODEL-DECLARED capability, `ModelFactory::consumes_multi_kv`, landed with its first consumer, and this architecture is that consumer. A step carrying all three published groups reaches the hook, which resolves every one of its five caches BY NAME through `MultiKvCacheIndex::Resolve` — including the recurrent members, which `ENG-MULTIKV-BYNAME` made addressable — and reads the QSA indexer side cache out of the engine's own group-2 pages through group 2's own gathered block table. The guard still refuses `DeepseekV4ForCausalLM`, `Glm5NextForConditionalGeneration` and every architecture that declares nothing, and clearing the bit drives that refusal red in the gate. **IT NOW DECODES, AND IT NOW SERVES** (W5k and W5L, #2031). This row said "it still decodes NO token, and the reason is now the MODEL and not the engine", and named a dtype and a residency: the recurrent group publishes the PLE conv ring at the model dtype while `RunQwen4ExpPleBlock` required f32, and publishes the n-gram token history as a device i64 state while the same block read it through a HOST pointer. W5k settled both against the RUNNING lane pin — transformers 5.16.0, `modeling_qwen4_exp.py` sha256 `77fec77d…c459`, confirmed by regenerating this row's committed forward golden byte-identically. Upstream types each cache slot from the tensor that first reaches it (`cache_utils.py:1019-1023`), so the ring carries the MODEL dtype and the history lives on the compute device: the PUBLISHER was right twice and both requirements moved to the block. `ModelRegistry::Forward` then runs a prefill and a `past_len > 0` DECODE over one set of persistent caches. **W5L drives the engine itself.** A real `GPUModelRunner` allocates all three published groups, gathers every group's block table, publishes the five-name by-name index and runs `execute_model` / `sample_tokens` for a prefill and then a decode; `LoadedEngine::FromModelDir` loads a `qwen4exp` GGUF and `generate` returns tokens; and `examples/server` answers `POST /v1/completions` on CPU. The cross-step gate is the PLE n-gram history read out of the RUNNER's own recurrent state at the slot the runner assigned — int64 token ids, which cannot saturate as this fixture's bf16 activations do. **WHAT SERVES IS EXACTLY THIS AND NO MORE: `--device cpu`, ONE SEQUENCE AT A TIME, over a GGUF.** `num_reqs > 1` is refused by name — `RunQwen4ExpQsaBlockPaged` takes a block table of one sequence — and because an EngineCore that meets that refusal dies rather than degrades, this factory sets `ModelFactory::serves_one_sequence_per_step` and `LoadedEngine::ResolveMaxNumSeqs` clamps `--max-num-seqs` to 1 and says so on stderr; concurrent clients are accepted and served in sequence. MEASURED before that clamp existed: three overlapping `/v1/completions` calls at `--max-num-seqs 4` each returned a 500 carrying this hook's own message, and the engine never served again. The quant arms are the loader's — IQ4_NL, Q5_0 and the dequantizing gather (#1989), whose CUDA arm landed with KGATHER. **ALL SIX `qwen4_exp` ops PLUS `vt::RmsNormGroup` NOW HAVE CUDA ARMS** (W6-CUDA and W6-CUDA-B, #2031) and this row's earlier sentence that "no CUDA arm exists for any `qwen4_exp` op" is false and is replaced. **THAT LEAVES THE GATHER, AND KGATHER LANDED IT**, so the sentence this row carried — that the one remaining reason is `EmbeddingKernelCuda` refusing a block-quantized table — is FALSE and is replaced. `vt::Embedding` on a CUDA queue decodes a block row across all 18 encodings `vt::cpu::BlockToFloat` decodes, measured bit-exact against the CPU arm on a GPU. **WITH BOTH LANDED, WHAT BLOCKS A CUDA FORWARD IS NEITHER OP REGISTRATION NOR THE LOADER, AND THIS IS A PREDICTION RATHER THAN A MEASUREMENT:** the expected shape is partial dispatch through the PLE, then a NAMED REFUSAL at the first QSA layer, because `qwen4_exp_qsa_block.cpp` still reads three operands on the HOST — `CheckRopeLayoutsAgree`, `IndexerRows` on the block table, and `Qwen4ExpQsaIndex` on `kv_lens` — which is owned by the QSADEV wave and NOT by this row. A second wall the synthetic fixture never reaches: `IsCudaKeepQuantSupported` still excludes IQ4_NL and Q5_0, which the released UD-IQ1_S uses, owned by [#2423](https://github.com/mudler/vllm.cpp/issues/2423). **A GPU HAS NOW PRODUCED TOKENS FOR THIS MODEL, ON ROCm, AND THE CLAIM IS LIVENESS AND NOT CORRECTNESS.** Measured 2026-09-13 on `strix:gpu0` (gfx1151, Radeon 8060S, ROCm 7.2.4) inside an `rc` lease ([spec](../.agents/specs/rocm-chunked-pinned-h2d.md)): the released `unsloth/Qwen3.8-Flash-Next-GGUF` UD-IQ1_S loads through `--device auto` and `examples/vllm-cli` returns 32 tokens with `finish_reason=length` on three of three launches -- fluent, coherent, prompt-dependent output, byte-identical across the three. Steady-state decode is **5.0-5.3 tok/s** -- nine samples across six independent process launches, 5.002 to 5.291 tok/s, 5.8% max-to-min. It is a RANGE: an earlier two-launch reading quoted 5.27-5.29 at a 0.34% spread and that precision does not reproduce. **IT IS NOT A TOKEN GATE AND IT IS NOT A PARITY CLAIM, AND THE REASON THIS ROW USED TO GIVE IS FALSE AND IS REPLACED RATHER THAN AMENDED.** It read "this architecture has no GPU oracle at all: llama.cpp aborts in `build_delta_net_chunking` before it reads a byte and no vLLM revision implements `qwen4_exp`, so nothing decoded these prompts beside us, there is no denominator". Only the STOCK `llama-cpp` pin `b10451` aborts there. The scoped [`llama-cpp-qwen4exp`](../.agents/oracles/llama-cpp-qwen4exp.md) oracle (PR #27742 at `035e22731a`, `gateable = yes`) builds with HIP for `gfx1151` and decoded this same UD-IQ1_S artifact on `strix:gpu0` at **25.877 tok/s** (median of 12 legs, 1.734% leg spread, [evidence](bench-evidence/qwen4exp-llamacpp-denominator-gfx1151-20260913.md)), and vLLM registers `Qwen4ExpForConditionalGeneration` at the ACTIVE parity pin `e126687a9a` (`registry.py:580`). **So a denominator EXISTS, and a primary oracle that could define token-exact EXISTS.** What is missing is a primary-oracle RUN: `cooperative_topk` in the QSA indexer refuses to launch on this fleet at that revision ([#2626](https://github.com/mudler/vllm.cpp/issues/2626)), every published safetensors arm exceeds the largest fleet box while vLLM cannot open the GGUF, and the decode GEMM plan is `sm_103`-gated. **The two figures are still NOT divided and no speed comparison is admissible here**, because `AGENTS.md` §Gates admits a performance result only after this arm's declared token-exact gate passes and it has none ([`ISSUE-LOCAL-01M2D6MV5RNSSM2GZVZKCZA4EG`](../.agents/issues/MODEL-MM-QWEN4-EXP/ISSUE-LOCAL-01M2D6MV5RNSSM2GZVZKCZA4EG.md)); the llama.cpp arm is also TEXT-ONLY while this row is a multimodal port. What reaches a token is the bounded pinned host-to-device ring; the same binary with `VT_ROCM_PINNED_H2D_MIB=0` stops at 29.69 GiB of device memory and produces nothing in 1200 s. **NO TOKEN HAS COME OUT OF A CUDA DEVICE FOR THIS MODEL and none is claimed** -- that half of the previous sentence is unchanged, and it is owed by the QSADEV host-operand wave and [#2423](https://github.com/mudler/vllm.cpp/issues/2423), not by the ROCm row. **NO TOKEN NUMBER AND NO SPEED NUMBER**: everything above ran on a synthetic fixture whose weights are a deterministic ramp, and the safetensors arm refuses because every published safetensors artifact exceeds every device this project owns. **THE CLAUSE 'no published `qwen4exp` checkpoint has been served' IS NOW HALF FALSE AND IS REPLACED BY A MEASUREMENT.** The released `unsloth/Qwen3.8-Flash-Next-GGUF` UD-IQ1_S (67.564 GiB, 3 shards, 1224 tensors) was driven through `examples/server` on `thor:gpu0` on 2026-08-30 (`rc` job `0f188dd1`, [evidence](bench-evidence/qwen4exp-released-checkpoint-serve-20260830.md)): **it LOADED and the server LISTENED, in 4446 s at 69.206 GiB peak RSS, keeping every one of its nine encodings quantized -- and it produced ZERO TOKENS.** `POST /v1/completions` returned 500 because the forward refused the artifact by name: `vt: qwen4_exp_gated_residual: input_mix_weight_down must be float (f32/bf16 for outputs)`. The file stores all **194** hyper-connection mix weights as Q8_0, the loader correctly keeps them quantized, and `vt::Qwen4ExpGatedResidual` accepted only float; every arm of the synthetic fixture wrote those same tensors as F32, which is why every gate on this row was green and none of them could see it. **W5p REMOVED THAT REFUSAL AT ITS SOURCE** (#2031): `mix_down`, `mix_up` and `block_inject` now accept a block-quantized `[N,K]` weight and route through `vt::MatmulBT`, which dispatches the keep-quant GEMM `kMatmulBTQuant` -- mirroring llama.cpp, which merged this architecture on 2026-08-27 (`6c84c7d5d`, first tag `b10660`), declares all six of these projections `GGML_OP_MUL_MAT` and never dequantizes one. The ELEMENTWISE operands did not move: a block-typed `hc_*_norm` gamma is still refused by name, which is llama.cpp's own split (`GGML_OP_MUL` for the norm, with an explicit f32 cast where a file-typed weight meets an elementwise multiply). The synthetic fixture grew the arm that was missing (`FixtureOpts::hc_mix_q8_0`), and `ModelRegistry::Forward` runs a prefill and a second prompt over a Q8_0-mix file; restoring the old contract reds that case with the verbatim string above, which is what makes the reach measured. **W5q RE-RAN THE RELEASED CHECKPOINT THROUGH THE REPAIRED PATH, AND THE REFUSAL IS GONE WHILE THE OUTPUT IS DEGENERATE** ([evidence](bench-evidence/qwen4exp-released-checkpoint-serve-20260831.md)): on `thor:gpu0` `--device cpu`, staged to worker-local disk, the artifact loads in **61 s** (against 4446 s from the CIFS share) at `VmHWM` 73.935 GiB, a 5-token prefill and eight decode steps run with nothing thrown, and `POST /v1/completions` returns **HTTP 200** with 8 tokens where W5n got a 500. **But every one of those tokens is id 0 — `!` in this file's own vocabulary — and the answer is BYTE-IDENTICAL for two different prompts.** So the forward is degenerate and prompt-independent on the real weights, no usable token has yet come out of a published `qwen4exp` checkpoint, and the DECODES and SERVES claims at the head of this row remain true OF THE FIXTURE. **W5s THEN GOT REAL TOKENS OUT OF IT, AND THE CAUSE WAS THE REPACK MARKER** ([evidence](bench-evidence/qwen4exp-released-checkpoint-tokens-20260831.md)): on `origin/main` `52f7ccbfc`, which carries W5r as well as W5p, the same artifact on the same box answers `" Paris. Given this fact, what is"` and `" 100°C at sea level"` — two different prompts, two different prompt-dependent completions, eight distinct token ids none of them 0. W5q's tree predated W5r, so on `thor` (aarch64 i8mm, where `vt::cpu::QuantRepackActive()` is TRUE) `dense_attn::ResidentWeight` was still dropping the repack marker and `kMatmulBTQuant` read `block_q8_0x4` buffers as flat `q8_0` on every hyper-connection mix weight; a read-only per-stage probe puts a NaN in `stream.after_layer_0` (`nan=51200`) collapsing to an all-zero `LOGITS` row (`zero=248320`), and `argmax` over a row with no maximum returns index 0. Post-W5r that stage is `nan=0` and the logit row is `min -9.89818 max 15.7873` with argmax id 11751 = the `" Paris"` token; `VT_CPU_QUANT_REPACK=0` is byte-identical to the default, which is what a correct performance transform must be. **WHAT RUNS ON A PUBLISHED CHECKPOINT IS EXACTLY THIS: the UD-IQ1_S GGUF arm, ONE SEQUENCE AT A TIME, on `--device cpu` -- and, for LIVENESS ONLY, on ROCm `--device auto` on one gfx1151 board as the paragraph above records -- and no more.** **IT IS NOT A TOKEN GATE** — no oracle decoded these prompts, the STOCK `llama-cpp` pin `b10451` aborts in `build_delta_net_chunking` before loading a byte (the scoped [`llama-cpp-qwen4exp`](../.agents/oracles/llama-cpp-qwen4exp.md) oracle does NOT, and it has since decoded this artifact on `gfx1151`), the other six published quants are unrun, and there is no speed number. The repaired route is also a per-TOKEN matvec where llama.cpp batches the projection over the whole prefill; batching it is owed and unmeasured. The shipped GGUF is TEXT-ONLY (1224 tensors, no `v.blk.*`), so the multimodal arm has no artifact to load either. **CONFIG LAYER GATED as well.** The config resolves and validates against a RUNNING transformers 5.16.0 oracle (it imports without torch, so `validate_architecture` executes): a 39-case two-direction sweep agrees on 35 and differs on 4, all four being local guards stricter than upstream, never looser. All 15 upstream `validate_architecture` rejections are implemented and tabulated against their upstream line. The forward and the KV-cache spec REFUSE BY NAME, each naming the wave that owes it. vLLM implements `qwen4_exp` at NO revision, so the algorithm oracle is transformers **5.16.0** under an accepted lane exception; `gateable = no` because nothing published fits a fleet device — `Qwen/Qwen3.8-Flash-Next` is ~360 GB bf16, ~180 GB FP8, ~128 GB NVFP4 against ~119.6 GiB usable on GB10 | none, and no speed claim is admissible from this row until a token gate exists | +| `Qwen4ExpForConditionalGeneration` | Qwen3.8-Flash-Next, [UD-IQ1_S GGUF](USAGE.md#checkpoint-registry), text-only | ⚠️ Prompt-dependent generation on [CPU](bench-evidence/qwen4exp-released-checkpoint-tokens-20260831.md) and [ROCm gfx1151](bench-evidence/qwen4exp-rocm-hcnorm-gfx1151-20260913.md). [CUDA generates too](bench-evidence/qwen4exp-cuda-decode-identifiers-20260902.md), but CPU/CUDA token disagreement remains unresolved. One sequence per step. No oracle token gate. | No accepted cross-engine comparison. [vLLM registers the architecture](../.agents/oracles/vllm.md), but its model run remains blocked. [The scoped llama.cpp oracle runs](../.agents/oracles/llama-cpp-qwen4exp.md). | | `Qwen3_5ForCausalLM`, `Qwen3_5MoeForCausalLM` | none: no text-only Qwen3.5 checkpoint fits this hardware | **NO RUN GATE, OWED.** Gated on `test_qwen3_8_text_only.cpp`; NO token claim. Loader reads stacked BF16 experts (#740) plus BF16 towers, shared expert and `lm_head` (#864), so both published indices satisfy the load plan | not measured | | `Qwen3ForCausalLM` | Qwen3 dense 0.6B/1.7B/4B/32B, NVFP4A16 | near-tie strict 16/16 vs vLLM 0.25.0 | c1 every-axis parity, c8 decode residual | | `Qwen3MoeForCausalLM` | Qwen3-Coder-30B-A3B | strict 6/6 vs vLLM 0.25.0 | 11/16 grid cells at or above graphed vLLM | From fb72c6a5e9c5a69b3b52199a4fecad9b2b9f8740 Mon Sep 17 00:00:00 2001 From: localai-org-maint-bot <306269227+localai-org-maint-bot@users.noreply.github.com> Date: Mon, 14 Sep 2026 03:14:58 +0000 Subject: [PATCH 3/3] record: capture the scoped documentation review Record independent review and the operator verification. Full preflight remains incomplete because the local tool environment cannot run its build and packaging checks. FOLLOWING_AGENTS_PROTOCOL Following-Agents-Protocol: true AI-Assisted: true Assisted-by: Codex:GPT-6 [Codex] --- .agents/specs/qwen4-exp-public-doc-refresh.md | 25 +++++++++++++------ 1 file changed, 18 insertions(+), 7 deletions(-) diff --git a/.agents/specs/qwen4-exp-public-doc-refresh.md b/.agents/specs/qwen4-exp-public-doc-refresh.md index 9b56c41d1..83558f26b 100644 --- a/.agents/specs/qwen4-exp-public-doc-refresh.md +++ b/.agents/specs/qwen4-exp-public-doc-refresh.md @@ -4,8 +4,9 @@ Documentation audit at base `cef9f8216`, 14 September 2026. No model lifecycle, runtime behavior, or benchmark acceptance changes. Implementation is complete. -Independent review and the fork pull request are pending. One pull request -carries the spec and documentation commits. +Independent scoped review passes at `f450f7626`. The operator reran the focused +gates. Full preflight is incomplete because of the tool environment, as recorded +below. One pull request carries the spec and documentation commits. ## Scope @@ -119,8 +120,18 @@ headings. In a scratch directory, replacing the README target with fragment with `#missing-anchor` also fails with exit 1. Restoring the original README passes. The source tree stays unchanged throughout both mutations. -Full `bash scripts/agent-preflight.sh` started before edits. Its log is -`/tmp/vllm-doc-preflight-impl-base.log`. At handoff, the full sweep is still -running. The operator owns completion and comparison with the untouched-base -log `/tmp/vllm-doc-preflight-base.log`. No GPU, runtime, or new benchmark gate -applies to this documentation-only correction. +Independent review of `f450f7626` found no issues. All seven changed link paths +and both fragments failed when corrupted in a scratch copy. Each byte-exact +restoration passed. The operator independently reran the four focused checks, +all 37 mutation cases, the link verifier, and the archive comparison. + +Full `bash scripts/agent-preflight.sh` was attempted on the baseline, +implementation, and review trees. The logs are +`/tmp/vllm-doc-preflight-base.log`, `/tmp/vllm-doc-preflight-impl-base.log`, and +`/tmp/vllm-doc-preflight-review.log`. The baseline and implementation encountered +the same nine failing checks. The review encountered five of those failures. +The registration checker cannot find CMake. Release subprocesses also remove +the library path that the temporary Python installation needs. The operator +stopped the incomplete sweeps after recording these failures. Full preflight +is **PENDING**, not green. No GPU, runtime, or new benchmark gate applies to +this documentation-only correction.