diff --git a/.agents/issues/_owed/ISSUE-LOCAL-01M221WAVSK2STJ49Y3WTDSDS3.md b/.agents/issues/_owed/ISSUE-LOCAL-01M221WAVSK2STJ49Y3WTDSDS3.md new file mode 100644 index 000000000..ec6575b5f --- /dev/null +++ b/.agents/issues/_owed/ISSUE-LOCAL-01M221WAVSK2STJ49Y3WTDSDS3.md @@ -0,0 +1,19 @@ +ID: ISSUE-LOCAL-01M221WAVSK2STJ49Y3WTDSDS3 +Title: Refresh the Qwen3.8 public documentation +Row: - +State: OPEN +Kind: docs +GitHub: - +Mirror: PENDING +Availability: FULL +Created: 2026-09-09 +Updated: 2026-09-09 +Closed: - + +## Problem + +README News omits the September Qwen3.8 EXL3 serving comparison. The benchmark index hides its incomplete repetitions, and the Qwen3.8 model guide contradicts its recorded FP8 token gate. + +## Resolution + +- diff --git a/.agents/specs/qwen38-public-doc-refresh.md b/.agents/specs/qwen38-public-doc-refresh.md new file mode 100644 index 000000000..63a62c3c2 --- /dev/null +++ b/.agents/specs/qwen38-public-doc-refresh.md @@ -0,0 +1,58 @@ +# Qwen3.8 public documentation refresh + +## Scope + +Correct public documentation against the code and committed evidence at base +`6db4bef90`. This is an editorial audit, not a feature lifecycle transition. +No model, kernel, test, or benchmark implementation changes are in scope. + +## Inventory and design + +| Surface | Gap | Required correction | Authority | +|---|---|---|---| +| README News | September EXL3 comparison is absent | Add a compact entry linking the measurement, with its incomplete and ungated limits | `docs/benchmarks/qwen38-27b-exl3-variadic-gb10.md` | +| Benchmark index | Mixed-length comparison reads only Measured | Expose incomplete comparator repetitions in its disposition | Same benchmark detail and committed run records | +| Qwen3.8 model guide | CUDA component paragraph denies a token gate documented earlier | Distinguish component validation from the later model comparison; simplify affected prose | `docs/models/qwen3-8-27b.md`, matching model and FP8 CUDA implementation, recorded gate evidence | + +## Verification + +Inspect the actual code for every changed behavior claim. Retain the recorded +hardware, oracle revision, workload, and limitations. Do not extrapolate a +sampled EXL3 serving comparison into correctness or speed parity. + +Run the existing README, benchmark-index, site, and agent-record checks, plus +staged preflight. No GPU, model download, external compute, or new benchmark +is needed or authorized. Runtime performance and correctness are unchanged. +A fresh agent implements these edits from this committed spec. A fresh reviewer +checks the immutable result against the code and evidence. The operator reruns +the focused gates before pushing to the fork and opening an upstream PR. + +## Risks and stop conditions + +Do not edit unrelated benchmark history or feature states. Do not replace a +historical measurement with an unmeasured current result. Report any unavailable +gate dependency explicitly. No merge into upstream is authorized. + +## Owed + +- ISSUE-LOCAL-01M221WAVSK2STJ49Y3WTDSDS3: correct the three public surfaces above. + +## Review handoff + +The editorial changes are implemented on the task branch. The canonical issue +stays open until upstream integration. No feature state or measured number +changes. Resume by reviewing the three public-file changes against these anchors: + +- `.agents/specs/gate-qwen38-27b-fp8-block.md:539`: model token gate. +- `.agents/specs/vt-matmul-fp8-block-cuda.md:638`: CUDA component evidence. +- `tests/vt/test_ops_matmul_fp8_block_cuda.cpp:9`: component test scope. +- `src/vllm/model_executor/models/qwen3_5.cpp:2652`: block-FP8 dispatch. +- `docs/benchmarks/qwen38-27b-exl3-variadic-gb10.md:13`: comparison limits. + +CPU-only validation uses the existing `check-readme-structure.py`, +`check-benchmark-index.py`, `check-site.py`, and `check-agent-record.py` scripts. +All four pass. The existing README and benchmark-index mutation suites pass. +Full preflight was attempted, but its initial run lacked Python validation +packages and native tool dependencies. Release packaging subprocess tests also +failed with the temporary Python runtime's shared-library lookup. Those results +are not a full-preflight pass. The focused editorial checks require no GPU. diff --git a/README.md b/README.md index b7a6a9998..5197fcc1c 100644 --- a/README.md +++ b/README.md @@ -37,6 +37,9 @@ ## News +- **2026-09** **Qwen3.8-27B EXL3 gets a mixed-length serving comparison on GB10.** + The [measurement](docs/benchmarks/qwen38-27b-exl3-variadic-gb10.md) covers concurrency 1, 4, and 8. + Comparator repetitions remain incomplete, configurations differ, and no correctness gate covers the run. - **2026-09** **C ABI 26 exposes more engine controls.** Applications can select the KV cache dtype, read speculative acceptance counters, and disable the model-level sliding window. See the [C API reference](docs/reference/c-api.md#recent-abi-additions) for defaults and limits. diff --git a/docs/BENCHMARKS.md b/docs/BENCHMARKS.md index 0faa76672..6358bbe0d 100644 --- a/docs/BENCHMARKS.md +++ b/docs/BENCHMARKS.md @@ -15,7 +15,7 @@ measurement evidence remains in [the benchmark record](../.agents/benchmark-reco | `speculative-decoding` | Speculative decoding measurements | Mixed | [Details](benchmarks/speculative-decoding.md) | | `qwen38-27b-exl3-gb10` | Qwen3.8-27B EXL3 3.5bpw with its DFlash2 draft, on GB10 | Measured | [Details](benchmarks/qwen38-27b-exl3-gb10.md) | | `qwen38-27b-q4km-gfx1151` | Qwen3.8-27B Q4_K_M on Strix Halo: llama.cpp, vLLM and vllm.cpp, under a failing token gate | Mixed | [Details](benchmarks/qwen38-27b-q4km-gfx1151.md) | -| `qwen38-27b-exl3-variadic-gb10` | Qwen3.8-27B EXL3 3.5bpw under a mixed-length serving load, swept over concurrency, on GB10 | Measured | [Details](benchmarks/qwen38-27b-exl3-variadic-gb10.md) | +| `qwen38-27b-exl3-variadic-gb10` | Qwen3.8-27B EXL3 3.5bpw under a mixed-length serving load, swept over concurrency, on GB10 | Measured, comparator repetitions incomplete | [Details](benchmarks/qwen38-27b-exl3-variadic-gb10.md) | | `tt-capture-default-decode` | Tenstorrent decode rate, capture default vs opt-out, on the P150 | Measured | [Details](benchmarks/tt-capture-default-decode.md) | | `how-we-measure` | Benchmark method and acceptance rules | Method | [Details](benchmarks/how-we-measure.md) | | `variadic-load-methodology` | How the mixed-length, swept-concurrency serving benchmark works | Method | [Details](benchmarks/variadic-load-methodology.md) | diff --git a/docs/models/qwen3-8-27b.md b/docs/models/qwen3-8-27b.md index 9d3a4c296..1c6fc977e 100644 --- a/docs/models/qwen3-8-27b.md +++ b/docs/models/qwen3-8-27b.md @@ -202,9 +202,10 @@ supported width to 576. The run reported 5 cases and 136 assertions with no failures and no portable-fallback line. Unsupported shapes returned their named refusal. -This is not a model gate and not a speed result. No token-exact comparison -against vLLM has run through this arm, and the device run recorded neither -controlled clocks nor contention. It proves only the shapes that were run. +This component run validates the seven tested shapes against the CPU reference. +It recorded neither controlled clocks nor contention, so it establishes no speed result. +The separate [model token gate](#the-token-gate-against-vllm) ran on 23 August 2026, +with six token-identical prompts and one adjudicated near-tie. [#1437](https://github.com/mudler/vllm.cpp/issues/1437) records both runs, milestone M5 of [#1189](https://github.com/mudler/vllm.cpp/issues/1189) owns the kernel, and [#1166](https://github.com/mudler/vllm.cpp/issues/1166) is the