fix(ENG-QWEN35-FULL-ATTN-STATE): validate state only for GDN consumers by VikashLoomba · Pull Request #3101 · mudler/vllm.cpp

VikashLoomba · 2026-09-09T02:22:38Z

Row

ENG-QWEN35-FULL-ATTN-STATE. Closes #3098.

What changed

Qwen3.5 models containing only full-attention layers loaded successfully but failed completion when the forward path validated unused recurrent state. Validation and input preparation now follow the loaded layers' actual Gated DeltaNet consumers. Both dense and MoE variants use the existing full-attention builder when recurrent state is unused. Real recurrent models retain their cache, metadata, and bounds checks.

The graph drivers ignore unused recurrent indices during padding and preserve generic shape checks. A dedicated C API regression exercises completion through the native ROCm providers. Production registry and graph tests cover both model variants. The existing runner regression now requires successful completion, attention-cache writes in both layers, valid sampling, and request feedback.

Before starting

The operator checked the code, source history, issues, and open pull requests, then reproduced the public failure. Specification commit f62e4d7db0a2e1fb7d0e0a3cb5dad7072f182690 precedes implementation. One pull request carries the specification, implementation, and test repair. The per-row inventory and evidence are in .agents/specs/qwen35-full-attn-state.md.

Evidence

At implementation head 776af2efd56327d193c3eba2b6934658ad7da1bc:

The independent public red failed after successful load with GDN state index out of range.
The operator's public green passed 3,364 assertions across 12 fresh engines. Both prompts produced four identical tokens in three repetitions of the default and callback sampling arms. Native embedding, cache writes, device sampling, and resolved buffer dtypes were observed.
The operator reran test_model_registry, test_qwen27_paged_forward, test_qwen35_paged_forward, and test_qwen3_5_decode_graph_seam: 86 cases and 4,633 assertions passed. The registry retains one existing skipped case.
Fresh review of the original implementation detected 43 of 45 injected defects and identified two missing query-offset witnesses. A separate implementer added those witnesses. The operator reproduced both guard-deletion failures for both graph drivers, and both AddressSanitizer controls passed.
The original implementation and query-offset repairs passed fresh review. Both independently rebuilt guard mutations failed as intended, and the sanitizer controls passed.
A separate implementer corrected the runner regression after the forward repair invalidated its former refusal expectation. The operator rebuilt and ran the runner suite: 41 cases and 1,914 assertions passed. Fresh static and mutation review passed for the runner change; restoring unconditional GDN validation fails the completion assertion for the original reason.
The operator's full preflight on this exact head passed, including all six affected translation units. Explicit CPU ISA and PR classification checks cover the applicable generic skips. ARM, CUDA, and Triton artifact audits do not apply to this change.

The spec retains exact commands, immutable source and executable hashes, fixture hashes, logs, and operator receipts. This change adds no public command or API signature.

Speed claims

No performance claim.

Honest gaps

The identical-GGUF primary-oracle gate remains PENDING. The pinned plugin rejects the actual text-only engine configuration before producing tokens. Successful local completion does not establish primary-oracle parity. The graph harness proves CPU routing and staging, without claiming GPU replay numerics. The row remains ACTIVE, and this MR is ready for review. Merging is the maintainer's action; the recorded local work is complete and the oracle gate above is the only item that prevents an oracle-parity claim.

FOLLOWING_AGENTS_PROTOCOL

Following-Agents-Protocol: true
AI-Assisted: true
Assisted-by: Codex:gpt-6-astra [Codex CLI]

A full-attention Qwen3.5 model loads but fails completion because the shared forward validates unused recurrent metadata against zero slots. Define the consumer checks, existing input-builder reuse, public red, pinned oracle workload, and mutation gates before implementation. This specification on row/ENG-QWEN35-FULL-ATTN-STATE tracks mudler#3098. Implementation and runtime verification remain pending. This commit changes no product code and makes no performance claim. FOLLOWING_AGENTS_PROTOCOL Following-Agents-Protocol: true AI-Assisted: true Assisted-by: Codex:gpt-6-astra [Codex CLI]

Full-attention Qwen3.5 models have no recurrent cache but receive legacy GDN metadata. Select validation and device preparation from the loaded layer consumers. Keep hybrid cache and metadata checks strict, and pass inert metadata into graph padding before copying indices. The operator reproduced the public red and verified twelve fresh-engine completions. Both registered siblings and graph drivers pass focused CPU gates. Thirty scratch mutations prove the tests detect removed guards and production calls. Pinned vLLM rejects the exact text-only GGUF configuration before an engine becomes usable. G4 stays pending in the row specification. This change makes no performance claim and tracks mudler#3098 on row/ENG-QWEN35-FULL-ATTN-STATE under draft PR mudler#3101. FOLLOWING_AGENTS_PROTOCOL Following-Agents-Protocol: true AI-Assisted: true Assisted-by: Codex:gpt-6-astra [Codex CLI]

The graph shape tests reject nonzero first offsets and descending internal offsets for both model siblings. Each case fails when its guard is deleted, so independent review can verify both offset guarantees. Production code remains unchanged. Record the repair evidence under issue mudler#3098. FOLLOWING_AGENTS_PROTOCOL Following-Agents-Protocol: true AI-Assisted: true Assisted-by: Codex:gpt-6-astra [Codex CLI]

…ssion The full-attention-only runner test still expects the former forward-side GDN refusal. Specify its successful execution and state checks before correcting the test. Keep fixture geometry and runner topology unchanged. Refs mudler#3098 FOLLOWING_AGENTS_PROTOCOL Following-Agents-Protocol: true AI-Assisted: true Assisted-by: AGENT:gpt-6-astra [Codex]

The reviewed consumer checks make the full-attention runner fixture succeed, so its old exception expectation now fails. Require completed execution, changed attention state, a valid sampled token, and request feedback. Preserve the existing fixture and keep GDN state absent. The focused case passes 18 assertions. All 41 CPU runner cases pass with 1914 assertions. Restoring unconditional dense GDN validation fails the corrected case at the original refusal. Product bytes remain unchanged. Refs mudler#3098 FOLLOWING_AGENTS_PROTOCOL Following-Agents-Protocol: true AI-Assisted: true Assisted-by: AGENT:gpt-6-astra [Codex]

Full-attention Qwen3.5 models have no recurrent cache but receive legacy GDN metadata. Select validation and device preparation from the loaded layer consumers. Keep hybrid cache and metadata checks strict, and pass inert metadata into graph padding before copying indices. The operator reproduced the public red and verified twelve fresh-engine completions. Both registered siblings and graph drivers pass focused CPU gates. Thirty scratch mutations prove the tests detect removed guards and production calls. Pinned vLLM rejects the exact text-only GGUF configuration before an engine becomes usable. G4 stays pending in the row specification. This change makes no performance claim and tracks mudler#3098 on row/ENG-QWEN35-FULL-ATTN-STATE under draft PR mudler#3101. FOLLOWING_AGENTS_PROTOCOL Following-Agents-Protocol: true AI-Assisted: true Assisted-by: Codex:gpt-6-astra [Codex CLI]

ROCm expands quantized embeddings because its operation table lacks a compressed gather. Register all 19 admitted codecs and decode selected rows directly into the caller's F32 or BF16 output. Keep the error record fixed at 16 bytes and validate full-width IDs before reading a packed row. Admit the existing Q8_K file geometry and distinguish embedding storage from matrix storage. The embedding role preserves packed decoder-only formats; the default materializer still requires a dot kernel and refuses embedding repacks. Bind allocation and launch to the queue's device. The gfx1100 operator passed all 15 final CTests and the required two-device witness. All 76 operation outputs meet the pinned oracle gates, including 68 byte-identical outputs. Matching traces preserve untraced outputs and show native selected-row decode with requested output and BF16 KV storage. The full staged preflight reports no failures and five argument-dependent skips. The separate CPU ISA audit passes. The row evidence names each skip and preserves the complete raw gate output. Keep the row ACTIVE. Original upstream fixtures await download authority. The pinned primary renderer refuses the exact bounded model, and both secondary models abort before tokens. Model token equality, matched oracle cache geometry, fresh mutation review, and reviewed-head operator gates remain pending. This change makes no performance claim. Refs mudler#3093. The branch retains the separately reviewed mudler#3101 prerequisite. FOLLOWING_AGENTS_PROTOCOL Following-Agents-Protocol: true AI-Assisted: true Assisted-by: AGENT:gpt-6-astra [Codex]

VikashLoomba · 2026-09-11T04:25:06Z

CI triage — every red check here is red in the same way on origin/main at 8df67640d, in main's own run 34538060205, and none of them touches this prerequisite's diff:

Check	Failure	Main evidence
`sanitize-cpu (address,undefined)`	`test_dots3_note_vision`, `test_dots3_note_audio`, `test_openai_api_server_dots3_mm_forward`	job 103074166410 fails the same three tests
`sanitize-cpu (thread)`	`test_gemma4_moe_fp8_native_arm_guard`	job 103074166517 fails the same test (numbered 162 there, 163 on this branch)
`windows-msvc-cpu`	`test_openai_api_server.exe` exits `-1073740791` (`0xC0000409`) at localiser cases 58/59/61	job 103074166371 is the same crash and exit code
`windows-msvc-vulkan`	the same crash and exit code	job 103074166311

build-test-cpu, build-test-vulkan and pr-size pass on this pull request. mergeStateStatus is UNSTABLE because of the four pre-existing checks above.

…this change lands The pull request body carries `Closes mudler#3098`, so on merge GitHub closes the mirror while `.agents/issues/ENG-QWEN35-FULL-ATTN-STATE/ISSUE-GH-3098.md` still read `State: OPEN` with a Resolution ending "Keep this issue open until qualified work lands". Local files are the issue authority and the GitHub copy is the mirror, so the two would have disagreed the moment this landed, and the policy requires the local record to carry dated resolution evidence when the work lands rather than leaving it for a later sweep. Closed through scripts/agent-issue.py so the local transition is written and validated before the mirror moves; the record now reads CLOSED, Mirror SYNCED, Closed 2026-09-11. No product code changes. FOLLOWING_AGENTS_PROTOCOL Following-Agents-Protocol: true AI-Assisted: true Assisted-by: AGENT:claude-opus-5-1m [claude-code]

…is branch still carries OPEN This branch carries the six ENG-QWEN35-FULL-ATTN-STATE commits that landed separately as mudler#3101. That squash rewrote them under one new sha, so the only collision is the canonical issue record: main reads State CLOSED because mudler#3101 discharged it, and this branch still reads OPEN. Main's copy is the correct one and is taken verbatim; no product file conflicts. FOLLOWING_AGENTS_PROTOCOL Following-Agents-Protocol: true AI-Assisted: true Assisted-by: AGENT:claude-opus-5-1m [claude-code] # Conflicts: # .agents/issues/ENG-QWEN35-FULL-ATTN-STATE/ISSUE-GH-3098.md

) ## Row `BACKEND-ROCM-QUANT-GATHER`. Refs #3093. ## What changed ROCm now keeps every admitted quantized embedding table packed and decodes selected rows directly into the caller's F32 or BF16 output. The provider covers all 19 codecs, preserves full-width ID checks, and binds the queue device before allocation and launch. The Qwen3.5 GGUF reader admits the existing Q8_K block geometry; its embedding materializer accepts decoder-only formats while ordinary matrix weights retain their dot-kernel requirement. Public load/completion tests reach every format and compare exact tokens and logits with same-binary dense controls. ## Before starting Issue #3093 tracks the missing ROCm quantized embedding registration. The committed spec and its reader/materializer amendments precede implementation. This row uses one pull request. Its spec owns the child inventory; the parent backend lifecycle remains unchanged. The branch includes the separately reviewed full-attention prerequisite from #3101. Its later runner test correction is imported byte-for-byte from `694f04525174d0c2dac8030ba8f1398f2683ec82`. The gather change does not modify that prerequisite's product implementation. ## Evidence The gfx1100 operator executed all 15 final CTests without a skip. The primitive suite passes nine cases and 11,784 assertions; the public suite passes 2,850 assertions. All 456 completion files equal the full trace capture byte-for-byte. All 76 operation outputs meet the pinned primary/secondary gates; 68 are byte-identical, with the remaining eight within the original primary tolerance. The applicable secondary GET_ROWS geometry passes six exact F32/BF16-RNE comparisons. The required two-device queue, bounds, and recovery witness passes. Matching rocprofv3 1.3.5 traces preserve each arm's untraced output bytes. Native operation allocations exactly match packed tables, I64 IDs, requested outputs, and constant 16-byte error buffers. The primary runs packed GPU selections followed by native decoders; its tensor memory counters are labeled separately from its profiler-visible allocator pool. A complete public trace passes 228 engines and 2,850 assertions, with BF16 KV reads/writes and exact token/logit repeats. The commands and their receipts are retained in `native-final-v4-commands.json` and `native-final-v4-operator-results.json`. The operator ran: ```sh ctest --test-dir build-rocm-quant-gather --output-on-failure --verbose -R '^(test_rocm_embedding_quant|test_capi_rocm_embedding_quant|test_gguf_keep_quant|test_ops_embedding_quant|test_gguf_dequant|test_backend_cross_device|test_gguf|test_rocm_arch|test_rocm_backend|test_rocm_fp8_kv_cache|test_ops_rocm_silu_rounding|test_exl3_rocm|test_gemma4_rocm_fp8_seams|test_backend_multidevice|test_runner)$' ``` The implementation head is `c5bb0f1f71f1db2bac5f436189a6a018fb62cbc3`. Exact-SHA path classification, commit style, and trailer checks pass. All 4,233 committed source, test, and build files match the frozen review manifest. The complete library builds cleanly with HIP and CPU-only configurations. The CPU runner suite passes 41 cases and 1,914 assertions. Exact recipes, hashes, source anchors, oracle outputs, raw traces, and operator receipts are in `docs/bench-evidence/rocm-quant-gather/`. - [x] `scripts/agent-preflight.sh --staged --quiet` exits 0 with no failures and five argument-dependent skips. CPU ISA passes separately; ARM, CUDA code-generation, and CUDA Triton artifact checks do not apply to this HIP build. - [x] Fresh review of the immutable implementation passed with no findings. The completed mutation runs detect all 44 injected faults, including all 19 codec changes, removal of production call sites, bounds handling, BF16 rounding, device selection, malformed metadata, and empty-table behavior. Four positive GPU controls pass. A separate reviewer build passes five CPU regressions; three accelerator-only controls explicitly skip in that CPU build. The reviewer and operator each ran the complete preflight on this head: all 31 affected translation units compiled. CPU ISA and PR classification checks cover the applicable generic skips. The operator then reran all 15 native CTests and the required two-device witness; all passed. All 456 completion captures and 81 generated corpus files match the previous frozen run byte-for-byte. - [x] The changed feature statement is scoped to the existing gather row in `docs/FEATURES.md`. ## Speed claims - [x] This pull request makes no speed claim. ## Honest gaps The bounded oracle comparison is qualified at this head: the comparator now rejects any primary record that carries an exception after generation and verifies the sealed model, configuration, prompt, repeat, dtype, and runtime identities plus the complete cache-view bounds before accepting a report. A focused validator test keeps the successful Q4_0 captures as its positive control and reddens when a post-generation exception is injected. A second review round then pinned every qualification guard with its own negative case on both the token and memory report paths (status-only refusal, exception and traceback shapes, wrong pins and runtime identities, wrong workload seals, FP32 dtype, an FP8 request, non-greedy sampling, a changed capacity contract, and invalid storage geometry), and registered the suite in the preflight SUITES array and the CI python lane so a guard regression fails a standing gate. The operator comparison of the 96-run primary matrix records 93 of 96 token-identical cases. The three IQ3_S first-prompt mismatches are an ORACLE-SIDE codebook difference, not a native defect: native IQ3_S decode is bit-exact with the format-defining llama.cpp grid, while the pinned vLLM gguf plugin decodes 150 of 2048 codebook byte positions at 62 where the grid has 15 (effective magnitude 15.5 vs 15, +3.333%). A CPU substitution experiment (plugin-codebook rows placed into an otherwise identical dense model) reproduces the primary's exact divergent tokens, and stock llama.cpp reproduces the native tokens byte-for-byte. The finding, receipts, and two-sided citations are recorded with the row evidence, and it is reported upstream as [vllm-gguf-plugin#129](vllm-project/vllm-gguf-plugin#129); that grid correction decides whether the three IQ3_S-p0 cases then pass or stay recorded oracle-blocked. The original-fixture coverage is satisfied at this head. All 32 upstream fixtures were downloaded with the developer's authority and verified at 289,655,872 bytes; their 160 exported tensors over the 16 primary codecs were compared native-versus-plugin and pass with 320 outputs, 140 byte-exact, and every remaining element inside the unchanged upstream tolerance (`atol=0.01`, `rtol=0.04`). IQ1_M and the F16 output case remain the documented exclusions, and Q8_K, MXFP4, and IQ1_XXXS stay with their separate secondary evidence. The comparison report now labels its own scope from the run rather than a fixed string; the label defect found during this round is filed and fixed in flow as #3126. This is ready for review. The exact Q4_0 model reaches the pinned primary renderer but it refuses `Qwen3_5TextConfig` where `Qwen3_5Config` is required, before tokens. The secondary llama.cpp oracles run through their committed one-line `s_copy` overlay and match native on all 12 bounded requests; the pristine binaries abort on their first decode, and the upstream runtime pads the requested 64-cell context to its own 256-cell minimum, so the matched-memory claim is scoped to the native/primary pair (both 65,536 bytes) and the secondary runs claim tokens and logits only. The original 32 upstream fixtures await authority for their 289,655,872-byte download. Generated operation tables do not replace those fixtures. The fork's real 2.4T model qualification remains separately owed under #933. The row stays ACTIVE until the remaining gates pass. No CUDA/HIP combined-device execution is claimed. Repairs after the review above, verified at this head. `d896102f1` re-homes three evidence paths `pr-size` could not classify: the run's capture archive moves up beside its seven sibling archives with its sha256 unchanged (`9f700062…a1bc7`, still the value `capture-manifest.json` seals), and the two minimal-diff `.patch` artifacts become documents that reproduce each diff verbatim, record the original file's sha256, and point at the sealed full-context archive members. `74baf8c08` replaces the stale assertion that Q8_K never appears in a GGUF file — the reader's `case 15` is correct and mirrors llama.cpp, which carries a real Q8_K traits row and validates its row data on load — with assertions that pin the reader geometry against `vt::BlockElems`/`BlockBytes`, the id round-trip, and the properties that actually distinguish Q8_K: no weight-side `vec_dot`, not keep-quant capable, and not taken by the generic composite. `79176b72e`, `873807215` and `fb4d78049` remove the same false claim from the four remaining comments that carried it, including the CUDA gather header that this branch is what made false, and repair the three line anchors this branch's own edits had made stale. A fresh reviewer mutated each guarantee in a scratch copy and returned PASS, including the classifier mutation that reproduces CI's unclassified-path error verbatim and the geometry, `HasQuantDotKernel` and restored-`CHECK_THROWS` mutations that each red the intended assertion; the operator independently reran the two CPU suites (10/10 cases and 6226/6226 assertions; 55/55 and 10350/10350). FOLLOWING_AGENTS_PROTOCOL Following-Agents-Protocol: true AI-Assisted: true Assisted-by: Codex:gpt-6-astra [Codex CLI] --------- Co-authored-by: Ettore Di Giacinto <mudler@localai.io>

…radicts the canonical record (#3144) `GATE-PR-BODY-TRAILERS`. Refs ISSUE-LOCAL-01M298Q7PPM145DQA5C30GRYJA. ## What changed `squash_merge_commit_message = PR_BODY` makes a pull request body the landed commit message, so `Closes #N` in a body closes the GitHub mirror the moment the squash lands. Local files under `.agents/issues` are the issue authority, and nothing checked that the branch also carries `ISSUE-GH-N.md` reading `State: CLOSED`. The two authorities diverge at the merge, and the authoritative half is the one left reading OPEN. `scripts/agent-pr-body.py` now refuses such a body. It already fetches the live body and already runs before a merge, which is exactly when this matters. ## Why here, and why only closing keywords `check-agent-record.py` cannot catch it: it is offline and never sees a body, and `check_canonical_issue_references` silently `continue`s on an unresolvable bare `#N`. Making that strict was the obvious repair and is the wrong one. Measured over four open pull requests, bare citations lacking a local record run 6 of 7, 4 of 4, 8 of 8 and 10 of 11 — so failing them would fire the gate on ordinary work, which AGENTS.md names as the defect rather than the discipline. The rule therefore keys on the closing keyword alone. ## Evidence Red before: a body closing an issue with no record exits 1 naming the missing path. Green after: a bare `Refs #N` still exits 0. Both are committed as cases in `tests/scripts/test_agent_pr_body.py` (suite: 21 passed, 16 subtests), and the spec's `## Tests` table is updated because that table is compared with the loaded suite rather than sampled. Found live on four pull requests in one review pass: #3101 carried a closing keyword for #3098, whose record read OPEN; #3095 did the same for #3092 while that row's own gate table reads FAILING; #3096 carries five closing keywords against four OPEN records and one issue with no local record at all; #3097 likewise for #3093, against a spec saying the row is not ready to land. On its first live run against #3095 it also caught a closing keyword for #3106, which the human review pass and I had both missed — that record reads OPEN and its Resolution says it stays open until the reviewed production repair lands. The spec's `Kind` line is amended in the same change, because it claimed this row edits no checker rule and now it does. FOLLOWING_AGENTS_PROTOCOL Following-Agents-Protocol: true AI-Assisted: true Assisted-by: AGENT:claude-opus-5-1m [claude-code] Co-authored-by: Ettore Di Giacinto <mudler@localai.io>

…ed records Main advanced 58 commits under this branch. Three files collided and each was resolved by the rule the record policy states rather than by a three-way merge. `.agents/issues/ENG-QWEN35-FULL-ATTN-STATE/ISSUE-GH-3098.md` is an add/add: main carries the CLOSED record because mudler#3101 landed that work, and this branch still carried its own OPEN copy. Main's is correct and is taken verbatim. `.agents/engine-matrix.md` and `docs/FEATURES.md` are keyed records, so an automatic merge is refused for them by policy. Main's complete version was taken and this branch's scoped edits were applied again on top: the `SPEC-MTP-GGUF` and `SPEC-DFLASH-GGUF` rows, and the one `GGUF F16 weights kept resident as F16` row. Verified afterwards that the result differs from main in exactly those three lines and that both files keep main's line count, so no unrelated key moved. FOLLOWING_AGENTS_PROTOCOL Following-Agents-Protocol: true AI-Assisted: true Assisted-by: AGENT:claude-opus-5-1m [claude-code]

## Row `BACKEND-ROCM-F16-WEIGHTS`. Refs #3092. Refs #3106. ## What changed ROCm retains supported dense and embedding GGUF weights in F16 storage. Each retained operand carries its resolved value dtype, so BF16 model arithmetic rounds the file values before multiplication or gathering. Ordinary NN/BT multiplication and both embedding ID widths reach native ROCm providers. Queue and graph ownership release transient conversion scratch without retaining a permanent converted weight copy. The default Qwen3.5 dense registry supplies the value dtype and rejects incompatible explicit model dtypes. Other loaders and stacked experts preserve expansion. This change integrates the reviewed full-attention consumer prerequisite from #3101 and its runner regression correction under #3098. It is independent of #2782 and changes no CI configuration. ## Before starting - The gap was verified at `6db4bef906859e864c82523c01107473f7dcca29` against the code, tests, issues, pull requests, and owning records. - One pull request carries specification `4486d24f07ce197155fa2b37c1ff590cdbe290e7` before implementation. The scratch-lifetime amendment also precedes its product changes. - The canonical child inventory and evidence are in `.agents/specs/rocm-f16-weights.md`. The child remains `ACTIVE`; the parent lifecycle does not change. - The design follows pinned vLLM `e126687a9a` and GGUF plugin `d4c1f0d082` through parameter creation, destination-dtype conversion, and native dispatch. ## Evidence CPU ownership and metadata tests, physical gfx1100 primitive and lifetime tests, graph dedup, and two-device controls pass. The new registry control passes 4 cases and 107 assertions. The complete CPU operand-contract suite passes 6 cases and 240 assertions. Guard-deletion mutations fail the new registry, embedding-provider, and NN-compute assertions; the unchanged controls pass. Scoped restoration covers absent, empty, and nonempty environment values and both provider states. The unchanged full-attention runner fixture now passes 18 assertions through successful execution, attention-cache writes, sampling, and request feedback. All 41 CPU runner cases pass with 1,914 assertions. Restoring unconditional dense GDN validation fails at the original refusal. The committed #3098 amendment precedes this test-only repair. `ctest --test-dir build-repair-cpu --output-on-failure -j 1` passes 703 cases, skips 12, and retains the single #3102 baseline failure out of 716. The operator runs the complete HIP suite under the GPU mutex with both devices visible, provider statistics enabled, and graph dedup disabled: 692 pass, 12 skip, and 20 fail out of 724. Every one of the 19 baseline failures matches the pristine diagnostic and assertion summary exactly. The remaining failure is #3106. The added public full-attention completion target passes. `bash scripts/agent-preflight.sh --staged` exits zero with every executed check passing and 678 translation units compiled. Its five argument-dependent skips are qualified by an explicit passing CPU ISA audit, passing PR #3095 classification, and inapplicable ARM/CUDA/Triton artifacts. Exact-range style, trailer, classification, Now, and diff checks pass on `9cf2bc9505fc377dd5ef707b15c1122e6f09c60d`. The spec records exact commands, source and binary hashes, skips, and baseline comparisons. Root verifies all 4,157 source files and 1,006 declared HIP inputs before and after the full run. The operator's full preflight on `9cf2bc9505fc377dd5ef707b15c1122e6f09c60d` also passed, including all 678 affected translation units. Fresh scoped review passed the three F16 guards, the prerequisite integration, and the runner regression. A separate implementer corrected the paged-attention source citation. Fresh record review passed on `4137b96369467e925bfdf0738e5bad013c89b58f`. All 7,274 other tracked files retain their tested bytes and modes. The operator verified the one-line delta and reran the applicable record, anchor, compilation-scope, and exact-range gates before publishing that commit. ## Repair round (heads `15d8e1682` and `6607faea2`) The committed spec and fix complete native unfused attention. Fresh review of `15d8e1682` returned PASS with no findings: four CPU guard mutations redden both model modes (default and explicitly unfused), and the operator's six GPU mutation gates each redden its guarantee on gfx1100: G1 unregisters the split and reproduces the recorded pre-fix refusal at the production regression (57 of 58, 10,457 assertions, `vt: no kernel for op AttnGateSplit`), G2 swaps the per-head query/gate layout, G3 replaces the nearest-even BF16 narrowing, G4 makes the gate widening lossy, G5 modifies the input, and G6 removes the current-device binding. Pristine rebuilt-head reruns of the five receipt gates exit zero. The operator receipts, the review report, and the mutation logs are retained with the row evidence, and the full preflight at `6607faea2` exits zero with the argument-dependent skips separately recorded. ## Speed claims This pull request makes no speed, latency, memory-ratio, or invocation-parity claim. The capacity-four materialized-oracle subcase matches all 256 generated IDs across both repeats. Matching rocprofv3 artifacts capture both executing paths, but native BF16-to-F32 outputs and atomics policy differ from the primary BF16-to-BF16 path. ## Honest gaps - F16-G3 is resolved at this head: the branch carries the committed `BACKEND-ROCM-ATTN-GATE-SPLIT` spec, the native ROCm `AttnGateSplit` kernel and its registration, and the model guards that reach it. The complete HIP GGUF test passes all 58 cases with 10,970 assertions, and #3106 closes when this change lands. - Native D1 repeatability and the primary arithmetic comparison remain failing under #2773. The identical GGUF primary remains pending under #3092 and the plugin's recorded limitations; #2624 owns the separate Qwen3.8 gateability measurement. The prerequisite's actual pinned engine attempt refuses `Qwen3_5TextConfig` before producing a token. - CPU token-anchor failure #3102 and HIP baseline failures #3070/#3105 retain their recorded diagnostics. Ownership does not waive any correctness gate. - F16 model activations and outputs, RMSNorm activation support, stacked-expert retention, and other model-loader admission are outside this row. Public checkpoint pins and refused arms are recorded in `docs/USAGE.md`. - The row remains `ACTIVE`. This session publishes for user review and does not merge. FOLLOWING_AGENTS_PROTOCOL Following-Agents-Protocol: true AI-Assisted: true Assisted-by: Codex:gpt-6-astra [Codex CLI] --------- Co-authored-by: Ettore Di Giacinto <mudler@localai.io>

VikashLoomba added 5 commits September 8, 2026 19:17

This was referenced Sep 9, 2026

feat(BACKEND-ROCM-QUANT-GATHER): gather packed embeddings on ROCm #3097

Merged

feat(BACKEND-ROCM-F16-WEIGHTS): retain F16 dense weights on ROCm #3095

Merged

VikashLoomba marked this pull request as ready for review September 11, 2026 03:34

localai-org-maint-bot mentioned this pull request Sep 11, 2026

fix(ENG-QWEN35-FULL-ATTN-STATE): validate state only for GDN consumers #3098

Closed

localai-org-maint-bot merged commit c316ada into mudler:main Sep 11, 2026
3 of 17 checks passed

localai-org-maint-bot mentioned this pull request Sep 11, 2026

feat(GATE-PR-BODY-TRAILERS): refuse a body whose closing keyword contradicts the canonical record #3144

Merged

Navigation Menu

fix(ENG-QWEN35-FULL-ATTN-STATE): validate state only for GDN consumers - #3101

fix(ENG-QWEN35-FULL-ATTN-STATE): validate state only for GDN consumers#3101
localai-org-maint-bot merged 6 commits into
mudler:mainfrom
VikashLoomba:row/ENG-QWEN35-FULL-ATTN-STATE

VikashLoomba commented Sep 9, 2026 •

edited

Loading

Uh oh!

VikashLoomba commented Sep 11, 2026

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

Conversation

VikashLoomba commented Sep 9, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Row

What changed

Before starting

Evidence

Speed claims

Honest gaps

Uh oh!

VikashLoomba commented Sep 11, 2026

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

VikashLoomba commented Sep 9, 2026 •

edited

Loading