feat(MODEL-MM-deepseek-v4-deepseek-v4-for-causal-lm): run the DeepSeek-V4-Flash-Vision tower on CUDA against llama.cpp b10766 - #3163
Open
localai-org-maint-bot wants to merge 113 commits into
Conversation
…ion support (#2411) DeepSeek publishes the vision checkpoint under the existing text architecture string, but neither pinned nor current vLLM implements its image path. A separate multimodal row keeps the text model's evidence intact while the processor, tower, merge, serving, and quantized arm are incomplete. The model-author runtime is the only complete vision reference. Pin it as a narrow secondary oracle with gateable set to no until a leased TP4 run loads the 156.287 GiB artifact and records generated output. FOLLOWING_AGENTS_PROTOCOL Following-Agents-Protocol: true AI-Assisted: true Assisted-by: AGENT:gpt-5.6-sol [OMP]
The task branch was pinned before concurrent changes reached origin/main. Merge the current base so the staged and commit protocol gates evaluate the tree that would land. FOLLOWING_AGENTS_PROTOCOL Following-Agents-Protocol: true AI-Assisted: true Assisted-by: AGENT:gpt-5.6-sol [OMP]
…and image processors (#2411) DeepSeek-V4 Flash Vision needs the model-author prompt and pixel layouts before later waves can attach the vision tower. This slice ports the pinned encoder, Pillow-compatible preprocessing, N-layout expansion, and named failures into the shared multimodal containers. The prompt encoder emits all six pinned task transitions. It rejects every non-null non-string or unknown task before rendering, while a null task remains absent. The input bridge checked-computes the required patch feature width. It rejects malformed shared image rows before extent checks or downstream indexing. Model inference does not reach W1 yet. W4 wires this processor into the registered model forward. W5 wires the runner, public ABI, and OpenAI server. Row MODEL-MM-deepseek-v4-deepseek-v4-for-causal-lm and issue #2411 own both waves. The focused gates cover every pinned encoder case and seven deterministic Pillow 12.1.1 resize fixtures. Sixty-two production-source mutations make every W1 guarantee red. FOLLOWING_AGENTS_PROTOCOL Following-Agents-Protocol: true AI-Assisted: true Assisted-by: AGENT:gpt-5.6-sol [OMP]
…OCm, and Vulkan gates (#2411) The developer requires every accelerator path to run on resource-controller devices. A CUDA pass cannot stand in for ROCm or Vulkan, and no direct SSH run can stand in for a lease. The spec now assigns separate CUDA, ROCm, and Vulkan waves. It pins the current fleet choices and records the measured RADV GFX1151 Vulkan device on strix:gpu0. FOLLOWING_AGENTS_PROTOCOL Following-Agents-Protocol: true AI-Assisted: true Assisted-by: AGENT:gpt-5.6-sol [OMP]
…tower and aligner (#2411) DeepSeek-V4 Flash Vision turns pixels into language rows in two stages that W1 could not reach: a 32-layer-capable native-resolution ViT and the downsample-3 aligner. This slice ports both as a config-driven composition over the public vt operations, so CUDA, ROCm and Vulkan run one implementation. The tower is non-causal over a whole image and carries no learned position table. Position comes from a 2-D RoPE whose height frequencies precede its width frequencies, exactly as the pinned torch.stack([hpos, wpos]).flatten(1) orders them. RMSNorm weights stay f32 and q/k widen to f32 across the rotary multiply, because the pinned module widens there and narrows once. The aligner zero-pads the patch grid to a multiple of three, unfolds non-overlapping 3x3 cells in F.unfold's [channel, dy, dx] order and projects them through GELU into the language width. A cached scalar gather maps every spatial pad cell onto one appended all-zero patch row, so the unfold needs no separate padded buffer. Reduced-shape goldens come from a direct transcription of inference/vision.py at the pinned revision under torch 2.11.0. They cover the RoPE tables, patch embedding, per-block outputs, final norm, unfold order and both aligner stages. Model inference does not reach W2. W3 owns weights, W4 wires the registered model forward and image-span visibility, and W5 wires the runner, public ABI and OpenAI server. Row MODEL-MM-deepseek-v4-deepseek-v4-for-causal-lm and issue #2411 own those waves, and the row spec lists them under Owed. check-attention-rung-consistency was red here. The tower's naive-rung reason sat 46 lines above its vt::Attention call and the checker reads 20. The reason now sits on the call. FOLLOWING_AGENTS_PROTOCOL Following-Agents-Protocol: true AI-Assisted: true Assisted-by: AGENT:claude-opus-5-1m [claude-code]
…t users run, and the two behaviours this spec missed (#2411) This spec was written on 2026-08-31, when no quantization of DeepSeek-V4 Flash Vision existed and no runtime outside the checkpoint repository could run it. Both facts changed, and the row was still planning against the old ones. The developer named unsloth/DeepSeek-V4-Flash-Vision-Exp-GGUF as the artifact that has to work. It ships the llama.cpp two-file shape, which this spec listed as a non-goal and a stop condition. That non-goal is withdrawn, with the reason recorded beside it: it was written to stop the row from asking users to reassemble weights, and producing a private combined artifact nobody downloads is now the thing that would do that. clip_mmproj_gguf.cpp already reads the two-file shape for Qwen3-VL, so no container is invented. UD-IQ1_S plus mmproj-BF16 is 77.65 GiB resident and fits one GB10, which makes the single-device gate reachable for the first time. The revision and four SHA-256 values are pinned because unsloth re-quantizes in place under unchanged names. llama.cpp merged deepseek4 vision on 2026-09-02. Release b10766 is the merge commit and the first release carrying tools/mtmd/models/deepseek4v.cpp, which returns 404 at the stock llama-cpp pin b10451, 315 commits behind. It is registered as llama-cpp-dsv4vision rather than by advancing the stock pin, because every floor already measured against b10451 means what that release does. It is not a second algorithm source; it is the first runnable reference for the exact artifact, on hardware this project leases, at a quantization our arm can match. Reading that implementation found two DeepSeek-V4 language behaviours that change under image input and that this spec never mentioned. The text GGUF's first shard holds 43 tensors and nothing else: one exp_probs_b_vl bias per language layer, the MoE routing bias for image tokens, which on the three hash layers replaces tid2eid routing rather than supplementing it. And the 128-token sliding window is not applied inside an image span. Both are exactly the failure risk 3 already names: drop either and the argmax stays plausible while the token gate stays green. They are specified, given tests that cannot pass on tokens alone, and listed under Owed. Removing the new AGENTS.md registry row makes check-oracle-pins.py red with "pinned in .agents/oracles/ but absent from the AGENTS.md table"; restoring it returns "oracle-pins ok (15 oracles pinned)". check-agent-record, check-symbol-anchors and check-now-current pass. No product code changes here. W3 owns the loader work, W4 owns the router bias and the window. FOLLOWING_AGENTS_PROTOCOL Following-Agents-Protocol: true AI-Assisted: true Assisted-by: AGENT:claude-opus-5-1m [claude-code]
The branch was pinned 1218 commits ago, which made agent-preflight.sh skip both commit gates because origin/main was not an ancestor of HEAD. Merge the current base so the protocol gates evaluate the tree that would land. FOLLOWING_AGENTS_PROTOCOL Following-Agents-Protocol: true AI-Assisted: true Assisted-by: AGENT:claude-opus-5-1m [claude-code]
… load the image-token routing bias DeepSeek-V4-Flash-Vision carries a second per-layer MoE routing bias, and this tree could not read the checkpoint that holds it. The GGUF arm's totality gate reported `blk.0.exp_probs_b_vl.bias` as a leftover its name map does not cover. The EXL3 safetensors arm refused `layers.0.ffn.gate.bias_vl` by name as a tensor no arm routes. The official dense arm accepted both and counted neither. The tensor is the bias the router adds when the token being routed is an image token. The pinned `unsloth/DeepSeek-V4-Flash-Vision-Exp-GGUF` `UD-IQ1_S` build carries one f32 `[256]` vector for every one of its 43 language layers, and its first shard holds those 43 tensors and nothing else. That was read from the shipped header on 2026-09-05 by an HTTP range request over the first 14 MB. Both weight arms now account for the tensor and load it beside the text bias, on every layer. The read sits outside the hash branch on purpose. A text token on a hash layer routes through `tid2eid` and takes no bias at all, which is why the converter emits no `ffn.gate.bias` there. An image token has no meaningful token id to hash, so on those three layers this bias is the only routing input it has. A read inside the `else` would drop them silently. The read is optional in both arms. A DeepSeek-V4 text checkpoint carries none of these tensors, so a required read would refuse every text checkpoint the tree already loads. llama.cpp marks the same tensor `TENSOR_NOT_REQUIRED` in PR #28154, at oracle `llama-cpp-dsv4vision`. Nothing selects the loaded bias yet, and this slice therefore lands unreached. Per-token selection between the two biases, the hash-layer replacement at forward time and the non-causal image-span window are W4's work. Row `MODEL-MM-deepseek-v4-deepseek-v4-for-causal-lm` owns that wiring, issue #2411 tracks it, and `.agents/specs/deepseek-v4-flash-vision.md` lists all three under `## Owed`. Refs #2411 FOLLOWING_AGENTS_PROTOCOL Following-Agents-Protocol: true AI-Assisted: true Assisted-by: AGENT:claude-opus-5-1m [claude-code]
…guage-side tensor delta at 43 names (#2411) W3B found and loaded the image-token routing bias, but nothing said whether it was the ONLY thing the vision checkpoint's language half needs. A port map that is complete by inspection is how a fourth tensor family gets discovered at load time, on a leased box, after W4 is written. Range-reading all three UD-IQ1_S shard headers of the pinned unsloth vision repository and diffing the 1371 names against scripts/dsv4_gguf_manifest_names.txt, the 1328-name TEXT manifest check-dsv4-gguf-namemap.py already pins, gives 43 names present in vision and absent from text, all of them blk.N.exp_probs_b_vl.bias, and zero names present in text and absent from vision. The two artifacts declare the same block count, expert count, hash-layer count and compress-ratio array, so the counts compare. The language half of this model therefore needs nothing from this port beyond that one bias, and the whole tower, aligner and sentinel group lives in mmproj-BF16.gguf, which W3A owns. W4 has no third unknown family waiting for it. The manifest is committed as a fixture rather than left in a session, because the checker that would read it cannot see this artifact yet: it generates 1328 expected names and asserts exact set-equality, so the vision artifact fails it by construction and no gate covers the shipped vehicle's language half. Extending it is a semantic checker change and needs its own red-before evidence, so it is owed rather than done here, with this measurement as the input. No product code changes. No correctness or performance claim. FOLLOWING_AGENTS_PROTOCOL Following-Agents-Protocol: true AI-Assisted: true Assisted-by: AGENT:claude-opus-5-1m [claude-code]
…ek4v mmproj into the W2 vision types (#2411) The published artifact users download is a llama.cpp two-file pair, and this tree could read only half of it. `unsloth/DeepSeek-V4-Flash-Vision-Exp-GGUF` ships a `mmproj-BF16.gguf` whose `clip.projector_type` is `deepseek4v`, and the `clip` mmproj reader's scope was `qwen3vl_merger` alone. W2 already has the tower and the aligner. This gives them their weights. Four layout mismatches separate what the file stores from what W2 consumes, and none of them crashes when it is wrong. Each one loads, runs, and produces a fluent, wrong model, so each one is derived from a pinned source rather than guessed, and each one has its own value-exact case. Attention is stored as three separate projections and W2 wants one fused weight. The fused row order is q, k, v because that is where `deepseek_v4_vision.cpp` slices the three back out with `RowSlice`. The MLP is stored as separate gate and up matrices and W2 wants one merged operand. Gate comes first because that is the half `vt::SiluAndMul` applies SiLU to, and it is the half the pinned converter's `gate, up = data_torch.chunk(2, dim=0)` took. The patch embedding is stored as a 4-D conv2d weight and W2 wants the 2-D torch Linear weight it came from; the converter produced the conv2d shape with a pure reshape of an `nn.Linear` over an `F.unfold`, so flattening it back is the identity in [channel, dy, dx] order and not a permutation. Every bias, every norm weight and the patch embedding are stored f32, which is llama.cpp's small-tensor convention rather than the checkpoint's dtype, so the linears narrow to the model dtype while the RMSNorm weights stay f32, exactly as W2's contract states. A reader that passed the file's dtype through would move twice the bytes on the model path and leave every token identical. The Qwen3-VL refusal keeps rejecting `deepseek4v`. `model_loader.cpp` goes straight from `RefuseUnsupportedClipMmproj` into `LoadQwen3VLVisionFromClipMmproj`, so widening that one refusal would route a DeepSeek projector into the Qwen3-VL reader. The new arm has its own refusal beside it, and a case asserts the old one still fires. The tensor map closes in both directions, for the reason the Qwen3-VL accounting already records: a projector whose extra tensors are dropped silently produces a tower that runs and is wrong. The enumeration returns 427 names at depth 32, which is the artifact's own tensor count. NOTHING REACHES THIS READER YET. `RefuseUnsupportedDeepSeekV4ClipMmproj`, `DeepSeekV4ClipMmprojVisionConfig`, `LoadDeepSeekV4VisionFromClipMmproj`, `DeepSeekV4ClipMmprojExpectedTensors` and `RefuseUnaccountedDeepSeekV4ClipMmproj` have no production call site, and the four sentinel vectors the reader returns have no consumer. Row `MODEL-MM-deepseek-v4-deepseek-v4-for-causal-lm` owns the wiring in W4, issue [#2411](#2411) tracks it, and the row's spec lists both gaps under `## Owed`. The real 934,462,656-byte file has also never been read by this code; the fixture is synthetic and built to the artifact's measured header, and W3 owes the first load. FOLLOWING_AGENTS_PROTOCOL Following-Agents-Protocol: true AI-Assisted: true Assisted-by: AGENT:claude-opus-5-1m [claude-code]
The row lands as one pull request, so the deepseek4v mmproj reader joins the branch that already carries W1, W2 and the spec amendment. FOLLOWING_AGENTS_PROTOCOL Following-Agents-Protocol: true AI-Assisted: true Assisted-by: AGENT:claude-opus-5-1m [claude-code]
The row lands as one pull request, so the image-token routing bias accounting joins the branch that already carries W1, W2, W3A and the spec amendment. FOLLOWING_AGENTS_PROTOCOL Following-Agents-Protocol: true AI-Assisted: true Assisted-by: AGENT:claude-opus-5-1m [claude-code]
The row branch was pinned before the oracle-pin record repair reached origin/main, so its tools suite still failed on an assertion main no longer carries. Merge the current base so the protocol gates evaluate the tree that would land. FOLLOWING_AGENTS_PROTOCOL Following-Agents-Protocol: true AI-Assisted: true Assisted-by: AGENT:claude-opus-5-1m [claude-code]
…e MODEL pin was bumped for (#2411) The W0 spec commit `b1122cf5a` took the MODEL matrix pin in `scripts/check-agent-record.py` from 379 to 380 and wrote its justification block, but it changed no test. `scripts/check-pr-size.py` refuses a checker change that carries no semantic mutation evidence in the paired suite, so the row gate reported `checker change 'scripts/check-agent-record.py' requires semantic mutation evidence in tests/scripts/test_agent_record.py`. That gate is CI-only, because it needs `--base` and `--head`, so `agent-preflight.sh` skips it and nothing local caught the omission. The evidence a bumped count owes is not the constant. Asserting the pin reads 380 restates the line the checker already holds and proves nothing about the tree. The invariant is that the pin and the rows agree, and that the row this bump was made for is one of the rows being counted. The new test names `MODEL-MM-deepseek-v4-deepseek-v4-for-causal-lm`, requires exactly one of it in `model-matrix.md`, and pins its lifecycle state. The hazard here is the opposite of the one the qwen4-exp and glm5-next tests guard. Those resist splitting one model into two or three rows. This row shares the architecture string `DeepseekV4ForCausalLM` with the existing text row, so a reader re-deriving the count from architecture strings reads it as a duplicate. Deleting it as one returns the tree to 379 and the pin follows, leaving the file internally consistent and silently short the entire image path. The test therefore asserts that both rows carry the string, that only the multimodal row qualifies it with `vision_n_layers=32`, and that the `deepseek-v4` lane holds exactly its four named rows. The running history in `test_model_row_ratchet_is_load_bearing` stopped at 375 while the pin had moved four more times. That history is what a later re-derivation reads to tell a bump that followed a row from a bump that silenced a failure, so it is extended through 379 -> 380. `.agents/engine-matrix.md` rides along because this change made it stale. The `ENG-RECORD-ANCHOR-RATCHET` row cites two line anchors inside the test file this commit edits, and inserting the test moved both by 76 lines. The row's own prose records this failure mode, and the two-way anchor ratchet turns it into a red gate, so the citations are re-pointed at the symbols they already named. Only the two line numbers change. FOLLOWING_AGENTS_PROTOCOL Following-Agents-Protocol: true AI-Assisted: true Assisted-by: AGENT:claude-opus-5-1m [claude-code]
…ower the Owed entry its commit body claimed (#2411) W2's commit body (`8c10cad16`) ends "Row MODEL-MM-deepseek-v4-deepseek-v4-for-causal-lm and issue #2411 own those waves, and the row spec lists them under Owed". The row spec did not. `## Owed` named W1's processors, W3A's mmproj reader, the `exp_probs_b_vl` trio, the GGUF name map and the three device lanes, and nothing at all about the W2 vision tower or the aligner. AGENTS.md, "Nothing lands dead", asks for the spec's `## Owed` specifically; `## Now` is a different surface with a different trigger, and it carried the only prose that mentioned W2 being unreached. The pull request body becomes the landed commit message here, so an unrepaired claim would have put a false statement on `main`. The entry now says what is unreached, names the row that owns the wiring and cites the issue. Measured rather than asserted. `grep -rn 'DeepSeekV4Vision\b'` over `src`, `include`, `examples` and `tests` returns no construction of the class outside `deepseek_v4_vision.cpp` and its test, and the only non-test file that includes the W2 header is `clip_mmproj_gguf.h`, which takes the config and weights types for W3A's reader and is itself already listed as unreached. The other three Owed claims on this branch were checked the same way and are true as written. W3A (`303e69108`) claims its five reader symbols and the four sentinel vectors are listed, and they are. W3B (`ebca4db83`) claims all three `exp_probs_b_vl` behaviours are listed, and per-token selection, the hash-layer replacement and the non-causal image-span window are each named. The spec amendment (`7fb23f584`) claims the two missed language behaviours are owed, and the same bullet covers both. FOLLOWING_AGENTS_PROTOCOL Following-Agents-Protocol: true AI-Assisted: true Assisted-by: AGENT:claude-opus-5-1m [claude-code]
…ng bias BYTES and refuse a narrow one
A fresh review of W3B (`ebca4db83`) found that the arm which previously refused
the vision checkpoint outright is the one arm whose loaded bytes nothing checks.
The EXL3 carried arm's case asserted the width of `gate_bias_vl`, that it was not
empty, and that `text[i] != vl[i]`. Two defects pass all three. A slot filled
with zeros keeps its width, and a SWAP of the two biases keeps them unequal. The
suite therefore proved the arm ROUTES the name while the commit and the spec both
claimed it LOADS the bias. W4 selects between the two biases per token, so a
transposition there routes every image token on the text bias and every text token
on the vision bias, changing which of 256 experts fire with no crash and no gate
red. That is the failure the spec's risk 3 names.
The case now asserts every element of `gate_bias_vl` on every layer, and every
element of `gate_bias` on the gated layer, against `dsv4_exl3_fixture::CarriedValue`
for the tensor's own name. Both reviewer mutations were reapplied and both are now
red. The all-zero `Account` plus `assign` mutation gives four failures reading
`CHECK( 0 == Approx( -0.999559 ) )`; the gated-layer transposition gives four more,
`CHECK( 0.0988742 == Approx( -1.24119 ) )` on the vision slot and
`CHECK( -1.24119 == Approx( 0.0988742 ) )` on the text slot. The loader was
correct for this finding and no product line changed for it.
The GGUF arm validated no width at all. `V4GgufCtx::Vec` checks the residency the
policy elected and the role a tensor was routed under, and no geometry, so a
`[E-1]` router bias published under an unchanged name loaded in silence. The
router indexes that vector by expert id, so the next reader walks off the end of a
short host `std::vector<float>` instead of getting a refusal. A checkpoint
re-quantized in place keeps its file name, which is why the porting rule asks for
a sha256 beside the repo id, so this is a live shape rather than a hypothetical.
The new `V4GgufCtx::Vec1D` takes the expected width and refuses. Red before: a
file whose KV declares `expert_count` and whose tensor is one narrower raised
nothing, and both new cases failed against an empty message. Green after:
`deepseek-v4 gguf: blk.0.exp_probs_b_vl.bias must be a 1-D [4] vector
(n_routed_experts), got rank 1 first dim 3`.
BOTH router biases are now checked, not only the vision one. The text
`exp_probs_b.bias` beside it carried the identical weakness. It is a long-standing
gap rather than a regression W3B introduced, and this widens the finding's scope
deliberately: repairing one while leaving its neighbour would leave the two to
drift the first time either is touched, and the safetensors arm already gets this
guarantee from `carried.Float(..., {ne})` while `glm5_next_loader.cpp` and
`glm_moe_dsa_loader.cpp` already get it from `LoadVecF32(g, name, e)`.
No arm requires the bias to be present on all layers or on none, and that is now
a stated decision rather than an accident. The choice is to MIRROR the oracle.
llama.cpp declares `ffn_exp_probs_b_vl` with `TENSOR_NOT_REQUIRED` for each layer
independently, so a partially converted file loads there, and refusing a file the
oracle accepts is a divergence that would need its own justification. This one has
none: the empty slot is a state the consumer must already handle, because a text
checkpoint presents it on every layer. W3B's "on every layer of a vision artifact
and on no layer of a text one" describes the two published artifacts and is not a
constraint the loader enforces. A new case pins the per-layer behaviour, so a later
change to an all-or-nothing refusal is a red test somebody has to argue with.
The spec's section 1 opened by calling the shard "the unsloth text GGUF's first
shard". The shard belongs to the language half of the VISION repository
`unsloth/DeepSeek-V4-Flash-Vision-Exp-GGUF`, and a genuine DeepSeek-V4 text
checkpoint carries none of these tensors, which is what the whole optionality
design rests on. As written, W4 could have read that paragraph as its brief and
concluded the opposite. The wording is repaired and the section is otherwise
unchanged.
`test_deepseek_v4_mm_loader` goes from 6 cases and 83 assertions to 9 and 105. The
other four suites hold their reviewed values: `gguf_load` 19 cases and 1056
assertions, `exl3_loader` 22 and 613, `moe` 12 and 716, `forward` 6 and 34. Every
mutated source was restored byte for byte and verified by SHA-256.
Nothing here selects the loaded bias. W4 still owns the per-token choice, the
hash-layer replacement and the image-span window, row
`MODEL-MM-deepseek-v4-deepseek-v4-for-causal-lm` owns that wiring, issue #2411
tracks it, and `.agents/specs/deepseek-v4-flash-vision.md` lists all three under
`## Owed`.
FOLLOWING_AGENTS_PROTOCOL
Following-Agents-Protocol: true
AI-Assisted: true
Assisted-by: AGENT:claude-opus-5-1m [claude-code]
… attn_qkv arm by name and bound the geometry
The header claimed that `gguf-py/gguf/tensor_mapping.py` maps
`vision.blocks.{bid}.attn.wqkv` to V_ENC_ATTN_QKV, "which the shared mmproj
base then writes as three SEPARATE `attn_q` / `attn_k` / `attn_v` tensors".
That mechanism does not exist at the pin. `gguf-py/gguf/constants.py` spells
V_ENC_ATTN_QKV `v.blk.{bid}.attn_qkv`, `conversion/base.py` contains no
occurrence of `qkv` at all, the only converter that splits a fused vision qkv
is the model-specific `conversion/qwenvl.py`, and
`conversion/deepseek.py::DeepseekV4FlashVisionModel.modify_tensors` splits
`mlp.w1` only. So a projector converted by the oracle's own
`convert_hf_to_gguf.py` carries the fused form and this reader cannot load it.
The shipped `unsloth/DeepSeek-V4-Flash-Vision-Exp-GGUF` mmproj carries the
SPLIT form, so the gap does not block the shipped vehicle. It does block
converting the checkpoint with the oracle's own script, which is a
quant-matched denominator W6 may need. The fused arm is therefore refused BY
NAME rather than implemented, and `## Owed` records it against issue #2411.
The refusal has to arrive before `RefuseUnaccountedDeepSeekV4ClipMmproj`.
That one reported "carries 64 tensor(s) that this build's deepseek4v reader
NEVER reads", which blames the artifact for a gap in this build and sends the
reader to re-convert a file that is already correct.
The geometry read from `clip.*` was also unbounded. `KvInt` widens every
integer spelling, so a signed `block_count` of -1 and an unsigned one of four
billion both reached `blocks.resize(static_cast<size_t>(config.depth))` on a
path that runs on a user-supplied `--mmproj`, and surfaced as `length_error`
or `bad_alloc` naming neither the file nor the key. Every field is now bounded
where the key that carried it can still be named.
The new block_count case asserts on the PARSED VALUE and not on the allocation
failing. A red-first case for an unbounded allocation performs the allocation
by construction: an earlier draft used four billion, asked for about 80 GB and
tripped the global Linux OOM killer twice on the development box. A test whose
only failure mode is `bad_alloc` is a crash, not a gate.
Red before: 18 cases, 16 passed, 2 failed, 2198 assertions with 7 failed. The
fused case failed on "NOT IMPLEMENTED", on "2411" and on the absence of "NEVER
reads". The block_count case failed on all four message assertions. Green
after: 18 cases and 2198 assertions pass. `test_clip_mmproj_gguf` stays 9 cases
and 272 assertions, because the Qwen3-VL arm is deliberately untouched.
Three more gaps are recorded and not fixed: the unkeyed vision `rope_theta`,
which llama.cpp hardcodes and the converter asserts nothing about; the four
`clip.vision.image_*` preprocessor keys the real file carries and nothing here
reads; and the `general.alignment` fallback the fixture never exercises.
FOLLOWING_AGENTS_PROTOCOL
Following-Agents-Protocol: true
AI-Assisted: true
Assisted-by: AGENT:claude-opus-5-1m [claude-code]
…, the frequency decay and the aligner row order measurable (#2411) W2's two golden fixtures are hidden 8 / heads 2 and hidden 16 / heads 4, so both are head_dim 4. That makes rope_dim 2 and leaves EXACTLY ONE frequency per axis, at exponent 2*0/rope_dim = 0, so inv_freq[0] is theta**0 = 1.0 for any theta whatsoever. Two mutations were therefore invisible: pinning `rope_theta` to a literal 10000.0 while a fixture declares 1234.0, and halving the exponent denominator, which is the classic transcription slip. Production is head_dim 64, i.e. 16 frequencies. The same reduction degenerated the aligner. Every fixture grid ((2,5), (3,3), (3,4)) aligns to ONE merged row at downsample_ratio 3, and with block_rows == 1 a row-major and a column-major walk of the merged grid are the same sequence, so swapping the two loops changed nothing. That order is not cosmetic: it fixes the spatial arrangement of the image tokens W4 hands the language model, and a 448x448 image (32x32 patches, 11x11 merged) would emit transposed rows with the token count, the dtype and every golden unchanged. The dedicated unfold case could not catch it either, because it builds its expectation with the same loop nesting as the implementation and is a tautology on this axis. A third fixture closes both. head_dim 16 gives four frequencies at exponents 0, 1/4, 1/2 and 3/4, and its theta 7919.0 is neither the default nor either other fixture's. Its grids 4x5 and 7x4 merge to 2x2 and 3x2, so both distinguish the two orders. One head is deliberate: heads 2 and 4 are already covered, and holding hidden_size at 16 keeps every stage inside the committed tolerances. REGENERATED WITH THE COMMITTED GENERATOR, which is the only option. There is no local checkout of the model-author pin and no network access to it, so the formulas cannot be re-derived from source; a transcription error shared between the generator and a new fixture would not be caught by adding fixtures from that generator. What can be checked was: the local torch is 2.11.0+cu130, the exact version the fixture records, and re-running the generator before this change reproduced the committed goldens BYTE-FOR-BYTE. The new file is a pure insertion of 7448 lines, so the two original fixtures are untouched. Two independent oracles say the row order is row-major, so the code is correct and was merely ungated; nothing here changes it. The model author's `inference/vision.py` is the algorithm oracle. llama.cpp release `b10766` = `9400c8946e4da5e7694f2c26d6d4e50e14b690fa` (oracle `llama-cpp-dsv4vision`) maps merged cell (r, c) to aligner row `r * n_llm_w + c` in `clip.cpp`'s `set_input` for `PROJECTOR_TYPE_DEEPSEEK4V`, and its graph in `tools/mtmd/models/deepseek4v.cpp` (blob `ffe8f59d9997` at that pin) reaches the same order through `ggml_im2col` over a [x, y, n_embd] tensor reshaped `[ne0, ne1*ne2]`. The new row-order case takes its destination index from that formula rather than from our loop nesting, so it is not a second copy of the implementation. THE GELU STAGE BOUND IS NOW DERIVED RATHER THAN DECLARED, and that is a finding rather than a concession. The declared 0.01f was LOWER than the 0.016f allowed for the `aligner_hidden` buffer that feeds it, which is not derivable: sup|GELU'| is about 1.0839, so GELU can amplify the error it is handed by about 8.4% and can never be relied on to shrink it. Measured per case, input -> output: 0.0078125 -> 0.0078125, 0.0078125 -> 0.0078125, 0.015625 -> 0.00878906, 0.0078125 -> 0.00390625, 0.0136719 -> 0.0117188. Every case ATTENUATES and none reaches the ceiling, so no divergence enters at this stage; the pre-existing `heads4_depth1` case already ran at 0.015625 against 0.016, one bf16 ulp from failing, which is how close the declared value always was. The derived bound is TIGHTER than 0.01f for three of the five cases, and `aligner_hidden` keeps its own absolute cap, so the stage stays transitively bounded at 0.0173f. Every stage upstream of GELU on the failing case is at or below what the pre-existing fixtures already produce: patch 0.00195312 against 0.004, vision and unfold 0.015625 against 0.024 where an existing case reaches 0.0234375, and aligner_hidden 0.0136719 against 0.016 where an existing case reaches 0.015625. The new geometry is not worse anywhere. Red before, green after, each mutation rebuilt (3 ninja steps) and restored byte-for-byte to sha256 6006b685da095ade85c2c353083860f81b04f183f3d0491bacc85899617b3aa0: pinning theta to a literal was GREEN before and now reds the 2-D RoPE case; halving the exponent denominator was GREEN before and now reds the RoPE case, the aligner_hidden bound and the new frequency-ladder guard; swapping the block_row and block_column loops was GREEN before and now reds the unfold golden, the output golden and the new row-order case at its first mismatched element. Two coverage guards keep the geometry from being reduced away again. One fails unless a fixture has at least four distinct frequencies and a non-default theta; the other fails unless a case has a merged grid larger than one in both dimensions. FOLLOWING_AGENTS_PROTOCOL Following-Agents-Protocol: true AI-Assisted: true Assisted-by: AGENT:claude-opus-5-1m [claude-code]
…repair evidence The five mutations that prove the new gate detects what it claims live only in this session's terminal until they are written down. The spec is where a later reader looks for what a wave measured, so the mutation table, the failed assertion counts and the before/after suite counts go here. The transpose row carries the reason its number is 132: that is 144 elements less the 12 on the diagonal, which is exactly the set a transpose can move. The index-0 check the repair replaced would have reddened on none of them, so the figure is what separates a value walk from a spot check. The section also records why the out-of-range block_count case uses 4096 rather than the four billion the defect admits. A red-first case for an unbounded allocation performs the allocation, and the first draft took the development box down twice through the global OOM killer. FOLLOWING_AGENTS_PROTOCOL Following-Agents-Protocol: true AI-Assisted: true Assisted-by: AGENT:claude-opus-5-1m [claude-code]
…on path's memory format and its per-layer pool traffic (#2411) Two W2 guarantees were stated in comments and measured by nothing. THE MEMORY FORMAT. `deepseek_v4_vision.cpp` says the tower is bf16 on the model path with two annotated f32 exceptions, and widening the attention-output buffer to f32 left the whole suite green. AGENTS.md says exactly why: a token gate CANNOT detect a dtype that is too wide, because the values still match while the path moves twice the bytes, and `vt::MatmulBT` and `vt::Add` accept the mix without complaint. The forward now reports the dtype of every internal scratch buffer it allocates, in allocation order, through an optional capture field that production never sets, and the test asserts the exact sequence. The two f32 entries are named and keep their reason: the pinned `apply_rotary` widens q and k before the f32 cos/sin multiply and narrows once afterward. The count of f32 entries is asserted from the other end as well, so a new wide buffer cannot be added without the case failing. THE PER-LAYER SCRATCH. The existing allocation case compares driver Alloc counts and pool statistics across two Forward calls at one shape, and a per-layer pooled Get/Put inside a single call reuses the same warm block and changes neither. Moving the `normalized` buffer into the block loop was therefore invisible, and at the production depth of 32 that form costs 32 extra pool round trips per image. Bounding `misses` after a single Forward, which the review proposed, does not reach it either, and that is measured rather than argued. One Forward from a drained pool, hoisted against un-hoisted: depth 2 17 gets (13 misses, 4 hits) vs 18 gets (13 misses, 5 hits) depth 4 21 gets (13 misses, 8 hits) vs 24 gets (13 misses, 11 hits) depth 8 29 gets (13 misses, 16 hits) vs 36 gets (13 misses, 23 hits) `misses` is 13 in both forms at every depth: the fixed working set is identical and the pool serves every extra request from its own free list. The observable that separates them is pool GET traffic PER LAYER, 2 hoisted against 3 un-hoisted. The case measures one Forward at two depths and asserts the slope is 2, the two buffers the shared MlpGateUpMethodBase seam legitimately owns (`UnquantizedMlpGateUpMethod::Apply`'s merged gate_up output and the activation it returns). Everything this file allocates is hoisted and contributes 0. The depth-independent `misses` bound is kept beside it because it is true and worth holding, not because it can see this. Red before, green after. Both mutations were GREEN on the tree before this change. Each was rebuilt (3 ninja steps, never 0) and restored byte-for-byte to sha256 6006b685da095ade85c2c353083860f81b04f183f3d0491bacc85899617b3aa0, and the tree was rebuilt and re-run green after each restore. Widening `projected` to f32 now reds the scratch-dtype sequence and the f32 count. Un-hoisting `normalized` now reds the slope; because the recorder names that buffer, the reviewer's literal edit no longer compiles and the mutation has to carry the record line with it, which also reds the name sequence. So the slope was proved load-bearing ON ITS OWN by a second mutation that adds one per-layer scratch buffer the recorder never sees: it reds `per_layer == kPerLayerPooledBuffers` and NOTHING else, 1 failing assertion in the whole suite. FOLLOWING_AGENTS_PROTOCOL Following-Agents-Protocol: true AI-Assisted: true Assisted-by: AGENT:claude-opus-5-1m [claude-code]
…borrow dropping the load-time repack markers (#2411) `BorrowResidentWeight` hands each block's gate-up weight to the shared MlpGateUpMethodBase seam as an OwnedTensor. It copied dtype, rank, shape, `nk`, bytes and `d_dev` and stopped, so `repacked`, `q8_0_aligned` and `elem_kn_repacked` were lost at the boundary. Those three say the BYTES were rewritten at load into a different block interleave or orientation while the byte count and the [N,K] shape are unchanged, so nothing downstream can notice: `ValidateTensor` does not check them and no value gate can see them. THIS IS THE DEFECT `main` ALREADY FIXED AT `7a937db8a` (#2031), re-introduced in a private helper. There the shared `dense_attn::ResidentWeight` built its tensor with `MakeTensor`, which never copies the flags, and an i8mm-interleaved `block_q8_0x4` buffer (136-byte blocks) reached `kMatmulBTQuant` flagged as flat `q8_0` (34-byte blocks). It decoded to NaN, then all-zero logits, then token id 0, with nothing logged, because the `lm_head` GEMM swallowed the NaN. That is the qwen4_exp degenerate output, and it cost a debugging campaign. PROPAGATE, DO NOT REFUSE. A fail-closed check here would reject a repacked weight rather than decode it, which removes the CPU i8mm fast path instead of fixing the loss; this tree has made that mistake before. `ResidentWeight` re-reads `repacked` and `elem_kn_repacked` off the OwnedTensor on its host-alias arm and keeps its own audit guard on the device-staging arm, so the markers only ever had to survive the borrow. NOTHING MOVES ON THIS HOST, and that is why it needs a structural gate rather than a numeric one. `vt::cpu::QuantRepackActive()` is true only on an aarch64 i8mm host, so on x86 both markers are always false and no golden can shift. W3A's mmproj reader is what makes it live rather than latent, because it can now hand this tower block-quantized weights. The gate therefore reads the markers the seam actually holds, through an observable accessor documented like the existing `cached_geometry_count`, and the forward is deliberately not run: this fixture's bytes are not physically [K,N], so a forward under `elem_kn_repacked` would read a genuinely mislabelled buffer. Three mutations, each applied, rebuilt (3 ninja steps) and restored byte-for-byte to sha256 6006b685da095ade85c2c353083860f81b04f183f3d0491bacc85899617b3aa0, with the tree rebuilt and re-run green after each restore. Dropping `repacked`, the state W2 shipped, reds two assertions; dropping `q8_0_aligned` reds two; dropping `elem_kn_repacked` reds two. Each marker is therefore individually load-bearing. An unmarked sibling block is asserted to come back unmarked, so a stub that answers true cannot pass, and the accessor's own range refusal is covered. ONE GAP STAYS OPEN AND IS NOT THIS ROW'S TO CLOSE. `ResidentWeight`'s device-staging arm returns `MakeTensor(w.d_dev.get(), ...)`, which carries no markers at all, so `q8_0_aligned` cannot reach a CUDA Q8_0 GEMM through that arm for ANY model. That is a shared-seam gap in `dense_attn_block.h`, not in this file, and it is recorded in the row spec rather than repaired here. FOLLOWING_AGENTS_PROTOCOL Following-Agents-Protocol: true AI-Assisted: true Assisted-by: AGENT:claude-opus-5-1m [claude-code]
…eader from repeating the split-qkv premise The file's opening comment listed the separate `attn_q` / `attn_k` / `attn_v` storage as a property of the format. It is a property of the shipped artifact. The header and the spec were corrected in the same repair, and a third copy of the retracted claim is how it gets read back in as true. The comment now also names the four refusals the suite gates, because a reader who only sees the four layout joins listed will not expect the refusal cases. FOLLOWING_AGENTS_PROTOCOL Following-Agents-Protocol: true AI-Assisted: true Assisted-by: AGENT:claude-opus-5-1m [claude-code]
…ion weight refusal so each one is load-bearing (#2411) `ValidateWeights` makes fifteen `ValidateTensor` calls and nothing reached any of them. Deleting the `aligner_w2_weight` refusal outright left the suite green, and the header's "RMSNorm weights stay f32" claim rested on three refusals nothing drove, so relaxing one to accept the model dtype was green as well. A refusal nothing reaches is not a contract, it is a comment. One table of 35 rows rather than 35 cases, and every row is individually falsifiable rather than collectively. Each of the fifteen tensors is offered with the wrong dtype and with the wrong shape; the two remaining `ValidateTensor` branches, a mismatched rank and absent storage, are driven on one representative tensor each. The dtypes are driven in BOTH directions. A model-dtype weight is offered as f32 and an f32 norm is offered as bf16, so the f32 polarity of the three RMSNorm weights cannot be flipped silently in either direction. Two blocks are covered rather than only block 0, because these checks sit inside a loop over every block, and a row that only ever looked at block 0 could not tell a loop from a single test. Red before, green after, each mutation rebuilt (3 ninja steps) and restored byte-for-byte to sha256 6006b685da095ade85c2c353083860f81b04f183f3d0491bacc85899617b3aa0, with the tree rebuilt and re-run green after each restore. Every one was GREEN before this change, and each now reds EXACTLY the rows that name it and no others: delete the aligner_w2_weight refusal 3 rows: its dtype, shape and storage relax norm1 from f32 to any dtype 2 rows: block 0 and block 1 norm1 dtype widen the MLP w1 shape to accept any 1 row: block 1 MLP w1 shape The third mutation is not one the review named. It was added because the first two only move dtype rows, and without it "each row is individually falsifiable" would be an untested claim about the shape half of the table. The row label is captured as a std::string rather than as the bare const char* it is stored as, because doctest stringifies a character pointer as a pointer and a failing row was unidentifiable. FOLLOWING_AGENTS_PROTOCOL Following-Agents-Protocol: true AI-Assisted: true Assisted-by: AGENT:claude-opus-5-1m [claude-code]
…ter-bias width before materializing it `V4GgufCtx::Vec1D` read the width off the tensor it had already loaded, so it dequantized first and refused second. For the `[E-1]` shape the two new cases feed, the order does not matter, because a bias one element short costs nothing to materialize. For a corrupt or absurd declared width it decides what the failure looks like: the loader would try the allocation the header asks for and surface `bad_alloc`, or get killed, instead of the named refusal. A guard whose only failure mode is a failed allocation is a crash rather than a gate, and an allocation sized from an unvalidated header takes the machine down with it rather than one test. A sibling suite in this tree fed an absurd `block_count` into an unbounded resize, reserved about 80 GB, and tripped the global OOM killer. The width now comes from the file header through `GgufFile::Get`, and the refusal happens before `Vec` is called at all. The message is unchanged, so both `NARROW` cases still assert on the tensor name and the owed width rather than on an allocation failing. Mutation, on this head: deleting the `VT_CHECK` and returning `Vec(name, role)` directly makes `dsv4 vision GGUF: a NARROW exp_probs_b_vl is REFUSED, not read past` and `dsv4 TEXT GGUF: a NARROW exp_probs_b is REFUSED too` fail together, four assertions against an empty message, which is the same red the guard was introduced against. The source was restored byte for byte, SHA-256 `8b282c47a11c3f3346001ab28678ba0195e42206b4941dbdabe1922b7ca4a565`. The five-suite gate passes under a 6 GB address-space cap. `test_deepseek_v4_mm_loader` holds 9 cases and 105 assertions, `gguf_load` 19 and 1056, `exl3_loader` 22 and 613, `moe` 12 and 716, `forward` 6 and 34. FOLLOWING_AGENTS_PROTOCOL Following-Agents-Protocol: true AI-Assisted: true Assisted-by: AGENT:claude-opus-5-1m [claude-code]
…width guard reads the header The repair evidence said `Vec1D` takes the expected width and refuses, and it did not say WHERE the width comes from. That is the part a later reader needs, because moving the check onto the loaded tensor would look like a harmless simplification and would quietly turn a named refusal into a failed allocation. The section now states the ordering, the reason for it, and the mutation that holds it. FOLLOWING_AGENTS_PROTOCOL Following-Agents-Protocol: true AI-Assisted: true Assisted-by: AGENT:claude-opus-5-1m [claude-code]
… repair evidence and size the geometry cache for W4 (#2411) The seven review findings and their repairs now have one place a reader can find them, with the numbers rather than the verdicts: the two fixture degeneracies and why a reduced fixture could not see `rope_theta` or the aligner row order, the llama.cpp confirmation of row-major, the measured per-stage error propagation behind the one tolerance that changed, and the pool measurement showing that bounding `misses` cannot see un-hoisted per-layer scratch. It also states the limit on the fixture provenance instead of leaving it implied. There is no local checkout of the model-author pin and no network access to it, so the goldens can only be regenerated with the committed generator, and a transcription error shared between that generator and a new fixture would not be caught by adding fixtures from it. What could be checked was checked and is recorded: the local torch matches the recorded version, and the generator reproduced the committed goldens byte-for-byte before the change. W4 GETS A NUMBER IT WOULD OTHERWISE FIND AT RUNTIME. The `IndexSelect` gather index is `aligned_rows * hidden_size * downsample_ratio^2` i32 per cached geometry. At the production hidden_size 1024 and ratio 3, a 73x73 patch grid gives 625 aligned rows and a 5,760,000-entry index, 23.04 MB, and `kGeometryCacheCapacity` is 8, so a full cache is 184.32 MB of gather indices alone. The other per-geometry tensors are small beside it, 1.36 MB of f32 RoPE and 21.3 kB of positions. W4 owns whether eight geometries is right for the image sizes the server admits and whether that index should be computed instead of cached. One shared-seam gap is recorded rather than repaired, because it is not this row's: `ResidentWeight`'s device-staging arm returns a tensor with no markers at all, so `q8_0_aligned` cannot reach a CUDA Q8_0 GEMM through it for any model. FOLLOWING_AGENTS_PROTOCOL Following-Agents-Protocol: true AI-Assisted: true Assisted-by: AGENT:claude-opus-5-1m [claude-code]
The row lands as one pull request. This brings in the four W3B findings: the EXL3 arm's loaded bytes are now asserted rather than only counted, both router-bias widths are refused before materialization, per-layer optionality is pinned as a deliberate mirror of the oracle, and the spec no longer says a text GGUF carries the vision bias. FOLLOWING_AGENTS_PROTOCOL Following-Agents-Protocol: true AI-Assisted: true Assisted-by: AGENT:claude-opus-5-1m [claude-code]
…evidence counts the repairs superseded (#2411) The W3B repair wave took `test_deepseek_v4_mm_loader` from 6 cases and 83 assertions to 9 and 105, and recorded the new figure in its own evidence section. The original W3B paragraph still stated the old counts as present tense, so the spec asserted two different sizes for one suite and a reader had no way to tell which was current. The repair agent could not fix this: the paragraph sits outside the two spec locations its scope allowed, and it correctly said so rather than reaching past its boundary. The old counts are dated rather than overwritten. They are the record of what W3B measured when it landed, which is what the evidence section is for; restating them as the repair's numbers would erase the wave's own result. FOLLOWING_AGENTS_PROTOCOL Following-Agents-Protocol: true AI-Assisted: true Assisted-by: AGENT:claude-opus-5-1m [claude-code]
The row lands as one pull request. This brings in the six W3A findings: the false split-qkv provenance is corrected and the fused attn_qkv layout the pinned converter actually emits is now refused by name rather than blamed on the file, the geometry read from clip.* is bounded, and the attention output projection, the square aligner linear, the shape guard and the use_silu branch each gained a case that a mutation reddens. FOLLOWING_AGENTS_PROTOCOL Following-Agents-Protocol: true AI-Assisted: true Assisted-by: AGENT:claude-opus-5-1m [claude-code]
…n capture refusals, the other half of the 21 ValidateTensor sites (#2411) The weight table landed one commit ago covers fifteen `ValidateTensor` call sites. There are 21. The other five sit behind `ValidateCaptureTensor`, and replacing that helper's entire body with a no-op left the suite GREEN: the stage goldens hand it CORRECT captures, so they exercise the happy path and no refusal at all. That is the same finding as the weight half and it is closed the same way. These refusals are gate-facing rather than production-facing, because production passes nullptr and copies nothing. That is the reason to drive them rather than a reason to skip them. A capture contract nothing checks lets a future parity gate hand the tower a wrongly shaped or wrongly typed buffer and then compare against whatever happens to be in it, which is a green that measured nothing. Fourteen rows: dtype and shape for all six capture tensors, the device check on one of them, and the block-capture count. The device row fabricates a tensor whose device field says CUDA without allocating anything, because validation throws before the pointer is ever read, so the branch is reachable on a CPU-only host. Red before, green after. Neutering `ValidateCaptureTensor` was GREEN before this change and now reds 13 of the 14 rows. The fourteenth stays green under that mutation and correctly so: the block-capture count is a separate check in `ValidateVisionIo`, not a `ValidateCaptureTensor` call, so a row that failed there would mean the table was testing something other than what it names. The mutation was rebuilt (3 ninja steps) and restored byte-for-byte to sha256 6006b685da095ade85c2c353083860f81b04f183f3d0491bacc85899617b3aa0, and the tree was rebuilt and re-run green after the restore. The spec also records that both new coverage guards were proved non-vacuous rather than assumed to be. A guard that passes because it asserts nothing is the same failure as the degenerate fixture it exists to prevent: setting the new fixture's theta back to the 10000.0 default reds the frequency guard at 0 >= 1, and replacing its grids with a single (2,5) reds the row-order guard. FOLLOWING_AGENTS_PROTOCOL Following-Agents-Protocol: true AI-Assisted: true Assisted-by: AGENT:claude-opus-5-1m [claude-code]
… the first device run of the vision arm `### W7-CUDA evidence` records both leases with their job ids, the sm_110 build proof (CUDA 13.0.88, 41 `.cu.o` all sm_110, `fa2: DISABLED` for [110]), the red, the fix, and the device block measured against W6's CPU block and against llama.cpp `b10766` directly -- judged against W6's declared bound rather than a number chosen here. `## Owed` is corrected where this wave falsified it. W4's windowed `dev_attn` refusal was recorded as UNMEASURED because no CPU build can execute it; it fired at `deepseek_v4.cpp:1319` and is now measured. Its image-span half still has not, and the two device routers' media refusal was never driven, so both are recorded as unmeasured rather than assumed to work. Four gaps this wave found are added. A served image on CUDA still cannot be answered: the blocker MOVED from vision residency to `vt: MatVec weight size mismatch`. `test_deepseek_v4_mm_chat`'s image branch encodes a CPU-only premise -- it asserts an error naming `W7-device`, which `ForwardDevice` can only emit when the device kernels are ABSENT -- so it cannot hold on any CUDA build. Eight `mm_reach` cases fail on aarch64 for a host-side quant-repack reason, PROVEN by an A/B on the same binary (repack on: 12 of 20 pass; `VT_CPU_QUANT_REPACK=0`: 20 of 20), so it is not a device failure. And `test_serve_deepseek_v4_mm` times out with no output, which nothing here explains. `## Now` is updated because it asserted that a CUDA build would still refuse an image step in the device routers, which this wave falsified. FOLLOWING_AGENTS_PROTOCOL Following-Agents-Protocol: true AI-Assisted: true Assisted-by: AGENT:claude-opus-5-1m [claude-code]
…DA review findings, and file what only prose recorded An independent review of the W7-CUDA wave (`4abe547d2`, `e0db328ca`, `c6ded626e`) confirmed the vision-tower staging fix and returned six non-blocking findings. This repairs them. The `dev_attn` comment said the spec records the windowed refusal as unmeasured, which the same wave had already falsified. It now says the refusal was measured on `thor:gpu0`, names `### W7-CUDA evidence` as where that measurement lives, and states that a lease run produced it rather than a committed test. `MakeTinyVisionTower` built ONE block, so the only cases that gate `EnsureResident` could not detect a staging miss on any block index above zero -- and the per-block loop is exactly where such a regression lands. The fixture now carries two blocks, each over its own arena slice, and the device case asserts every block rather than the first: after staging it destroys the host arena and runs the SAME staged tower again, so a weight that was never copied changes the second answer. A new case that needs no CUDA forbids the fixture from degenerating back to one block; with `depth` returned to 1 it fails `REQUIRE( 1 >= 2 )`. The stale-cross-reference record claimed four citations of `deepseek_v4.cpp:413` and the tree carries three: `deepseek_v4_weights.cpp:1068` names the anonymous message with no line number. The count and the list are corrected, and the three real citations now read `:504`, where the guard sits. Two statements are made exact rather than dropped. The probe's W6 arm produces byte-identical `.f32` artifacts and a stdout that now carries one extra `provider:` line, which is what "byte-unchanged" was being used to mean. And the f32 arm refuses a device BEFORE it writes any artifact, so a refused run can no longer leave one valid-looking file for a later compare to consume. Three things this wave's green does NOT cover are recorded under `## Owed` instead of being implied: no committed test drives the windowed `dev_attn` refusal, so its MEASURED label is history; making `EnsureResident` a no-op leaves the whole CPU family gate green, so no committed gate protects the staging fix; and `dsv4v_w6_compare.py` carries no bound and emits no verdict, so the 4.9% / 0.998 judgement is prose arithmetic a reader performs on its output. Four defects the wave found lived only as spec prose under the umbrella issue. Each now has a canonical local issue under `.agents/issues/MODEL-MM-deepseek-v4-deepseek-v4-for-causal-lm/`: the `vt: MatVec weight size mismatch` blocker a served CUDA image meets, the `mm_chat` image branch whose expectation can only hold on a build without device kernels, the aarch64 quant-repack failures proven by an A/B on one binary, and the `test_serve_deepseek_v4_mm` timeout whose cause is not guessed. A fifth, under `MODEL-DSV4-EXL3`, records that `test_deepseek_v4_exl3_forward_loop_arm` is the one ctest entry of 27 in this family with no build target behind it. Gate: the CPU family gate is 25 of 25 passed with `test_cuda_deepseek_v4` skipped at 77; 25 of the 27 registered tests had a binary present, the two absent being that loop-arm entry and `test_serve_deepseek_v4_mm`, which was not built here. No GPU was used and no lease was taken. Issue #2411. FOLLOWING_AGENTS_PROTOCOL Following-Agents-Protocol: true AI-Assisted: true Assisted-by: AGENT:claude-opus-5-1m [claude-code]
…le spec anchors, and gate the per-block vision staging on a CPU host Second-round review of 4ccee2f40 found two defects in this row's records, and one of them was a claim that a gate was impossible when it was not. THE ANCHORS. 4ccee2f40 added six comment lines above the windowed `dev_attn` guard, which moved it from `deepseek_v4.cpp:1319` to `:1325` and made the spec's two citations of it point at a comment. Re-grepping every `file:line` citation in the spec found three more that were already stale: `deepseek_v4.cpp:4345` for the `kDevicePending` refusal, which is at `:4415`; `model_registry.h:299-356` for `MultiModalForwardInput`, which is at `:397-403`; and `inputs.h:20-92`, whose four named members span `:20-136`. All five are corrected and each is re-verified against the tree. `api_server.cpp:373` and `test_deepseek_v4_exl3_forward.cpp:443,446` were checked and are ACCURATE, so they are left alone, the second belonging to `MODEL-DSV4-EXL3` in any case. The `:2958` and `test_deepseek_v4_mm_reach.cpp` citations sit inside verbatim historical logs and are deliberately NOT renumbered, because renumbering a pasted log would falsify the evidence it records. THE GATE, which is the substantive half. A `## Owed` entry recorded that a per-block staging miss in `DeepSeekV4Vision::Impl::EnsureResident` could be caught only on a leased device. That is false. `EnsureResident` keys on `queue.device != weights.device` and stages through the backend the tower was CONSTRUCTED with, and neither is a CUDA predicate; `deepseek_v4_vision.cpp` resolves no backend from the registry at all. `vt::Queue` is a plain aggregate and `vt::Backend` has six pure virtuals, so a hand-built non-CPU queue and a host-memory fake backend run the staging loop on an ordinary CPU box. `test_deepseek_v4_vision.cpp` now counts the staged allocations and copies through the production `DeepSeekV4Vision::Forward`: 15 at depth 1 and 23 at depth 2, which is `2 + 8*depth + 5`, with the per-block slope asserted separately. The forward refuses by name straight after staging because the patches stay on the host, and that message is asserted, so a case that stopped reaching the staging loop cannot pass quietly. PROVEN to catch the defect rather than merely to run. Making the loop skip blocks with index > 0 rebuilt clean (rc 0) and moved the binary md5 from cc44ef7cd0d230402de70219cf54d9d2 to df9c8239f73e67734116968e303c9b7a, and the case then read `test cases: 1 | 0 passed | 1 failed` on `CHECK( 15 == 23 )` and slope `CHECK( 0 == 8 )`. Restoring the file byte-for-byte returned sha256 c614f8174e8982ddd8b4a51e0a8f94005aa7c0b9981257d155ebae205f05dab4 and, after forcing a recompile, the md5 and the green. The first restore attempt did NOT rebuild, because `cp -p` preserved the old mtime and ninja skipped the translation unit; the verdict was re-taken only after the md5 was proven back. The old entry's true half is kept: a no-op `EnsureResident` leaves every other CPU case green, because it returns on its first line for a CPU queue with host weights. The leased-device gate for a REAL device copy is still owed. A third entry said no committed test sets `VT_V4_DEVICE_ATTN`. Literally true of ctest, but misleading: `tools/parity/dsv4v_w7_cuda.sh:207` drives that refusal and greps for it. The entry now names that harness and keeps what matters, that it is not ctest-registered and needs a manual lease run. Gate on this aarch64 CPU host, VLLM_CPP_CUDA=OFF: test_deepseek_v4_vision 16 | 16 passed | 0 failed, 7420 assertions, up from 15 and 7412. No GPU was used and none of this needed one. 1 of the 26 registered deepseek_v4 ctest entries has a binary in this build, because one target was built by name; the other 25 would report ctest "Not Run" rather than a pass. FOLLOWING_AGENTS_PROTOCOL Refs: ISSUE-LOCAL-01M29PH41YKXY2EF05AVRH96QC Refs: #2411 Following-Agents-Protocol: true AI-Assisted: true Assisted-by: AGENT:claude-opus-5-1m [claude-code]
…m's vision accounting and make W6 able to fail A fresh review of the safetensors port and the W6 parity harness found one ungated product path and four harness or record defects. All five are repaired here. THE EXL3 ARM'S OFFICIAL-VISION ACCOUNTING WAS GATED BY NOTHING. The block in LoadDeepseekV4Exl3 that requires each expected vision tensor, routes it and counts it could be deleted whole, and both loader suites stayed fully green -- 16 of 16 and 22 of 22 -- while its dense twin reds exactly one case. Nothing reached it because every committed vision fixture used OfficialVisionOptions, the released vehicle's DENSE shape, so no case ever handed the loader an exl3 quant_method checkpoint that also carried the vision group. Exl3VisionOptions supplies that fixture and one case drives the production loader entry through it, requiring has_exl3_weights so it cannot pass against the dense twin and checking the vision group is counted exactly once. Deleting the block now reds it, with the arm's own by-name refusal as the message. THE PARITY DRIVERS RECORDED TEE'S EXIT STATUS. Under `set -u` with no pipefail, `cmd | tee f; step name $?` records the status of tee, which succeeds whenever it can write the file. The worst instance was the image generator, the single input the whole gate rests on: a failure recorded `image RC=0` and the sweep went on against whatever img392.rgb a previous run had left in a persistent NAS directory nothing clears. The three drivers take pipefail, the image step reads PIPESTATUS and then refuses on failure or on an empty image, and the stale image is deleted first. Compounding it, every driver appended a status to steps.txt that nothing ever read back, so all three ended on their DONE banner at rc 0 no matter what failed; each now reads it back and exits non-zero naming the failing steps. THE COMPARATOR ENFORCED NO BOUND. dsv4v_w6_compare.py returned 0 for every shape that matched, whatever the magnitude, so a run drifting from 3.83% to 12% would have produced a well-formed report, RC=0 and no signal. It now reads its bounds from a committed record -- kept out of the comparator so that no wave can derive a bound from the run it is judging -- and exits 1 when the bound is exceeded and 3 when no rule matches the tag, because an unjudged leg must never read as a pass. AND THE BOUND ITSELF CLAIMED MORE THAN IT MEASURES. W6's condition 3 bounds the shipped bf16 path by 3.34% + 1.57% = 4.9%, but both addends were measured in the same session as the 3.83% being judged, and condition 2 independently establishes 1.34% against a 1.57% floor, so the triangle inequality already forces 4.68% < 4.9%. Condition 3 is close to an identity given the other two, not an independent test. It is kept, because a gross regression still trips it and it is now machine-enforced, but the spec says plainly that it may not be quoted as independent evidence and that conditions 1 and 2 carry the result. Two downstream sites that quoted 4.9% carry the same qualification. Two smaller repairs ride along. docs/FEATURES.md still said the real mmproj-BF16.gguf had never been read, which the row's own spec falsifies at the same head. And the W6 input-stage comparison rearranges the oracle's buffer into OUR claimed patch order before writing it, so it asserts the ordering rather than measuring it; the code and the spec now say so, and the block-level permutation check remains the gate's actual ordering evidence. src/ and include/ are byte-identical to the base commit: the only compiled file this change touches is the test. Three harness weaknesses a reviewer flagged without mutating -- a guessed vit re-layout the report does not record, a normalisation the probe hardcodes instead of taking from the shipped processor, and a best_match that does not require distinct reference rows -- were neither reproduced nor repaired here. They are recorded in an open row-owned issue rather than left in prose. FOLLOWING_AGENTS_PROTOCOL Following-Agents-Protocol: true AI-Assisted: true Assisted-by: AGENT:claude-opus-5-1m [claude-code]
…parity harness unable to pass when it should fail eb6009f made dsv4v_w6_compare.py judge a leg and gave three drivers pipefail and a steps readback. A fresh review found seven ways the result could still report success over a failure. Every one is reproduced on synthetic data with the exit code before and after, and none was argued from reading. THE COMPARATOR. A non-finite judged statistic is now a failure, and a judged profile refuses a degenerate row. One image row that is zero on BOTH sides made stats() return nan for that row, the mean over rows nan, and both `nan > 0.049` and `nan < 0.998` False, so a single row silently disabled both magnitude bounds for all 100. Measured: 99 rows at 50% off plus one zero row printed VERDICT PASS and exited 0; it now exits 1 naming `degenerate` and `NOT FINITE`, while the same data without the zero row still fails at 49.9835% and a clean 2% leg still passes. An all-zero aligner cell is treated as a defect rather than an average to absorb, because it means the tower produced nothing for that cell. A missing or non-boolean `judged` key, and a tag mapping to a profile the file does not define, are ERROR on the new exit 4. `profile.get("judged", False)` read an incomplete profile as DIAGNOSTIC and exited 0 on 50%-off data. Only an explicit `judged: false` is a diagnostic leg now. Stages a profile declares are required to be PRESENT and are judged. Only image_rows was ever judged, so an absent vit, a vit 100x wrong and a shape-mismatched input each printed VERDICT PASS; each now exits 1. The spec cites the vit numbers as evidence that the error does not jump at a stage, and that sentence rested on nothing executable until this existed. No fresh literal was invented: the vit bound is built exactly as the existing cells bound is, from the same recorded measurements at the vit stage, 2.07% + 1.00% = 3.07%, measured inside it at 2.45%, 1.872% and 1.51%; the f32 arm gets the 1.00% vit floor against its measured 0.88%. The `cells` STAGE is diagnostic_only with its reason, being the same aligner rows image_rows already covers in pre-layout order, and the CLI leg gets its own block-only profile because llama-mtmd-cli writes only the block, so requiring stages there would fail a correct leg. THE DRIVERS. All four read their steps back against an EXPECTED list. `awk '!/ RC=0$/' | wc -l` counts an ABSENT step as zero failures, so a steps.txt holding only configure and build, with every comparison never having run, gave BAD=0 and exit 0; an empty file did too. A line no readback can parse is also refused. dsv4v_w7_cuda.sh was never repaired and it is the driver that produced the 2.884% / 0.99939 / 1.872% figures the spec quotes. It carried `set -u` with no pipefail, wrote a bare `cpu_control identical` into its own steps.txt, never read steps.txt back, and ended on `### W7_CUDA_DONE` with no status derived from any step. It now takes pipefail, records a real RC= line per step including the control, checks every copy, reads its steps back, and cannot reach its DONE banner when a step failed. ONE JUDGEMENT CALL, stated because it is not mechanical: a device REFUSAL on sm_110 is recorded under its own step name rather than as a failing step. thor is outside the vendored FA-2 arch set, so an FA-2-gated path can only refuse there and the refusal is the result that job collects; making any non-zero step fail would fail the driver on the box its own header says can only refuse. An unexplained non-zero exit with no refusal message in the log is still a failure, so the classification cannot absorb a crash. The three byte-for-byte controls the file headers themselves call load-bearing each record a step and fail the job on mismatch; they printed a message and recorded nothing, so a build that did not reproduce the recorded block still ended green. Every cp whose result matters clears its destination first and records a step, so a failed copy can no longer leave the previous run's file for the comparator to judge out of a persistent NAS directory nothing clears. EVIDENCE. tests/scripts/test_dsv4v_w6_compare.py, 18 cases, registered in agent-preflight.sh SUITES and green there. It executes each driver's OWN readback block, extracted from its EXPECTED line, rather than a copy of the logic. Mutation-proven rather than asserted: reverting the comparator to eb6009f changed its md5 from 71c8988c to c2636b3b and turned 6 cases red; reverting dsv4v_w7_cuda.sh changed its md5 to 1ffe1431 and turned 8 red; both were restored and cmp reports them byte-for-byte identical with the suite green. WHAT IS NOT CLOSED. The W7-CUDA job has NOT been re-run under the repaired driver; that needs a thor:gpu0 lease this wave did not hold, and it is recorded under the spec's `## Owed`. The 2.884% / 0.99939 / 1.872% figures are NOT withdrawn: they were produced red-first and that job's CPU control reproduced W6's block byte for byte. What the Owed entry records is that the harness around them could not then have reported a failed step. Also recorded on ISSUE-LOCAL-01M29SAM8CKS2XQM8EPEEN9Z67, which stays OPEN: its item (1) is fixed, the vit re-layout now reports `oracle_relaid_out` so a reader can tell whether the numbers came from the file or from the guessed re-indexing; and its item (3) is now MEASURED rather than plausible, because fixture rows built from a smooth ramp were near-parallel enough that bf16 rounding noise made best_match's argmax arbitrary and the identity permutation read 17 of 100 on three datasets including a clean one. Closes ISSUE-LOCAL-01M29ZEWHZZ8A97B9JKAE171QN FOLLOWING_AGENTS_PROTOCOL Following-Agents-Protocol: true AI-Assisted: true Assisted-by: AGENT:claude-opus-5-1m [claude-code]
…dging key, and judge the W7 steps against what the record says they do The previous repair made `judged` mandatory and left the same shape on every other judging key: they are read with `profile.get(...)`, and a bound whose limit is None is not applied, so a judged profile that OMITTED a key was judged without it. Measured on data whose every image row was 50% off, tag lp0, profile shipped_bf16: dropping `mean_rel_l2_max` exited 0 VERDICT PASS, dropping it with `mean_cos_min` exited 0 PASS, and a profile holding `judged` alone exited 0 PASS. Fixing that per key is what produced this commit, so it is fixed as a class: the judging keys are declared once in PROFILE_KEYS and STAGE_KEYS, validate_profile() checks the resolved profile before anything is judged, and a missing, mistyped or wrongly typed key is ERROR with exit 4. A bound this project decided not to apply is written as null and names its reason under `unbounded`, so _check_bound can reach limit=None only through a declaration. Each of those four cases is now exit 4, and so is dropping any one of the seven keys individually; the clean leg still exits 0. Three consequences of that file becoming complete rather than partial. A diagnostic_only stage is still required to be PRESENT: the flag was tested before the presence check, so an absent `cells` stage exited 0 PASS against a bounds file that says its presence is reported, and it is now 1 FAIL. The f32 arm's `input` stage is declared PRESENCE-ONLY with its reason instead of being bounded by nothing while its prose implied otherwise; no recorded f32-arm measurement exists at that stage, and removing a key from that rule now exits 4 where it exited 0 PASS. And the comparator narrates what it judged, one JUDGED line per bound, stage and presence-only rule, because a verdict that does not say what it covered cannot be read for what it left out. The W7 driver could not fail on a crash and would have failed on a clean run. Its refusal classifier matched the bare string `must be`, which ordinary assertion text carries: `Assertion failed: n must be positive` and a std::out_of_range `vector index must be less than size` were both filed as the expected device refusal with RC=0, and both are UNEXPLAINED now that every alternative is anchored to a message the product owns. In the other direction its readback required every step to be zero, which this row's own recorded thor run does not satisfy: ctest_cuda is non-zero for the aarch64 i8mm repack failures, the test_serve_deepseek_v4_mm timeout and the mm_chat CPU-only premise, and dev_attn_on is non-zero because the device decode path refusing is the POINT of that step. Read back over that recorded run the old block exits 1 with `### FAILING STEPS` and no banner, and the new one exits 0. The classification is not an excuse: an unattributed suite, a non-zero naming no failing test, a non-zero with no log, and a dev_attn_on that PASSES each fail. The three W6 drivers printed their `### W6_*_DONE` banner and then exited 1, so a log grep for the banner read a failed run as finished; the exit comes first now and the test executes each driver's own readback rather than asserting a substring is present in the file. best_match returns the margin between the winner and the runner-up, and every judged profile declares `best_match_margin_min`, so the identity-permutation condition the spec leans on for ORDERING evidence carries the separability precondition it never had. That 0.01 is DECLARED, not recorded, and its provenance says so: the degenerate ramp measures 1.47e-09 and the fixture 0.229, while no real leg has reported a margin because the statistic did not exist until now. Mutation-proven rather than asserted: deleting the max_degenerate_rows refusal from judge() reds test_the_degenerate_row_rule_is_load_bearing on its own, and deleting the margin check reds the near-parallel case; both files were restored byte-for-byte, proven by cmp and md5 10d99232afe1aad76d6a548dc81617de, with the suite green again. The degenerate rule cannot be isolated through a data file at all, because stats() returns nan for any zero-norm row and the isfinite guard then carries the case, so that test judges a synthetic report with finite means. tests/scripts/test_dsv4v_w6_compare.py goes from 18 to 42 cases, rc=0. No C++ is touched. The W6 and W7-CUDA jobs have NOT been re-run under the repaired harness, which needs a lease this wave did not hold. Closes ISSUE-LOCAL-01M2A27NZR23T89WXRGX2RNN5C ISSUE-LOCAL-01M29SAM8CKS2XQM8EPEEN9Z67 stays OPEN: its item (3) is fixed by the margin gate here, and its item (2), the probe's hardcoded f32 normalisation, is untouched. FOLLOWING_AGENTS_PROTOCOL Following-Agents-Protocol: true AI-Assisted: true Assisted-by: AGENT:claude-opus-5-1m [claude-code]
…mbership, and stop judging the device refusal from a prefiltered excerpt A fifth fresh review of the W6/W7 parity harness found one silent fail-open and three false-red routes. Each is fixed here and each is demonstrated on a concrete dataset with the exit code on both sides, the OLD side being the harness as it stands at 5c2cf83. A STAGE DROPPED FROM A PROFILE WAS JUDGED BY NOTHING. PROFILE_KEYS forced the `stages` KEY to exist and constrained its MEMBERSHIP not at all, so deleting a whole stage RULE was the one way left to drop a bound while declaring nothing: every ordinary judging key had to be written or the run was ERROR. Measured on a leg whose vit file was 100x wrong, dropping the `vit` rule printed `stage 'vit': NOT REQUIRED by this profile`, VERDICT PASS and exited 0, and `"stages": {}` did the same, while the identical data with the rule present exited 1 at 9900.3609%. validate_profile() now requires a judged profile to name every stage in STAGES, and a stage it does not require is written as null with its reason under `unbounded` as `stages.<name>`. Dropping a rule is 4 ERROR, an emptied map is 4 ERROR, a bare null is 4 ERROR, and a declared null passes while printing the reason. shipped_bf16_block_only carried `"stages": {}` and now declares all three stages null, because llama-mtmd-cli writes only the block. THE DEVICE REFUSAL WAS CLASSIFIED FROM A PREFILTERED EXCERPT. dev-attn-refusal.txt is a grep of dev-attn-on.log piped through `head -20`, and it discards two of the three refusal families the classifier is anchored to. Measured: a log holding `DeepSeek-V4 vision compute dtype must be bf16` classifies rc=0 as the full log and rc=1 as the filtered file, which is 0 bytes, and the kDevicePending text does the same. Driving the shipped readback block with such a log went rc=1 with `### UNEXPLAINED dev_attn_on` and W7_CUDA_FAILED, and is now rc=0. This would have fired on the next thor lease. The excerpt is kept for a human reader and is marked as judged by nothing. THE REFUSAL ANCHOR COVERED 23 OF 61 MESSAGES WHILE CLAIMING TO COVER ALL. The comment called `DeepSeek-V4 vision .*must ` the whole Invalid() vocabulary, "27 distinct messages". Counted in deepseek_v4_vision.cpp: 61 distinct `DeepSeek-V4 vision*` literals, 23 matched and 38 did not, including every ValidateTensor label at :103-146 and every overflow refusal. A probe leg refused by one of those was recorded `_unexplained` and failed the job. The anchor is now the product's own prefix, and across all 366 emitted forms the rejected count goes from 38 distinct literals to 0. The round-4 fix is NOT undone: `must be` is not re-admitted, and all five crash strings are still rejected, because no assertion or standard exception carries this product's message prefix. THE MARGIN BOUND WAS A DECLARED CONSTANT, AND IT WOULD HAVE RED CORRECT RUNS. best_match_margin_min was 0.01 and the bounds file said in its own provenance that the number was declared rather than recorded. Measured with the shipped best_match() at realistic width (D=1280; the fixture uses D=16), an ordinary photographic geometry falls under it: a shared global component with 10% and 5% per-cell detail gives margins 0.00834 and 0.00210, and a 20-cell flat region gives 0.00875 and 0.00221, while the identity stayed best for 100 of 100 rows in all four. min_best_margin is a MIN over the cells, so one flat pair decides a run. The key is replaced by best_match_margin_above_bf16_rounding, and the margin is judged against the run's own bf16_rounding_scale, derived from the rows it read as the largest direction change half a bf16 ULP can cause. The five photographic geometries go 1 FAIL to 0 PASS and the degenerate ramp stays 1 FAIL, 1.47e-09 against 7.51e-06. The scale is built from the bf16 GRID SPACING rather than the residual `x - bf16(x)`, which is exactly 0 for data already on the grid, as both sides are by the time they reach a file. The run also now says what KIND of bound it applied. The suite goes from 42 to 48 cases at rc=0, and red-first is proven as a suite rather than asserted: the new test file run against the ORIGINAL 5c2cf83 tools/parity gives 23 failures and 2 errors covering every new guarantee. Four further review findings are FILED rather than fixed, to keep this wave scoped: ISSUE-LOCAL-01M2A5T6AP63M5F3D9CT9NBPKX (stale spec line anchors in shipped comments), ISSUE-LOCAL-01M2A5T6N98WABAAXZH4D52CRR (three malformed bounds inputs crash instead of reaching exit 4) and ISSUE-LOCAL-01M2A5T6ZPD6QXSV3Q3MSAXVJB (a stray `NN - <text>` line inside a ctest FAILED block). A fourth, ISSUE-LOCAL-01M2A5T7B187YJ9GMFSWAGB20P, is closed because this change falsified it. Closes ISSUE-LOCAL-01M2A5QKV6H5038W2M9BG9K7MG. RE-RUNNING THE W6 AND W7-CUDA JOBS UNDER THE REPAIRED HARNESS IS STILL OWED: it needs a thor:gpu0 lease this wave did not hold, so no real leg has yet reported its own min_best_margin or bf16_rounding_scale. FOLLOWING_AGENTS_PROTOCOL Following-Agents-Protocol: true AI-Assisted: true Assisted-by: AGENT:claude-opus-5-1m [claude-code]
Main gained 11 commits while this row was in flight, including the PR-body trailer gate (a253247, b7fb4e5) and two DeepSeek-V4.1 rows. One file conflicted: .agents/model-matrix.md, a keyed record. It was resolved by taking origin/main's version whole and reapplying only this row's edits, never the three-way result: the architecture row and the MODEL-MM inventory row, plus the five counters that one ACTIVE row moves (ACTIVE 13->14, Total 381->382, rollup 381->382 and "four"->"five", engaged 57->58). The 356 + 21 = 377 architecture split and the qwen4exp/glm5next reconciliation paragraph are main's, unchanged. Verified: the state histogram sums to 382 and matches the printed Total; the engaged table holds 58 data rows against main's 57; every line main added to this file survives byte-identical except the Total it superseded; and the diff against origin/main is exactly the five intended counter lines. FOLLOWING_AGENTS_PROTOCOL Following-Agents-Protocol: true AI-Assisted: true Assisted-by: AGENT:claude-opus-5-1m [claude-code]
…gration Main gained 15 further commits while this row was being gated: the QUANT-GGUF-IQ4_NL ROCm work, BACKEND-TENSTORRENT-KEEPQUANT W4d, and a re-derived q4km Tenstorrent capture golden (#2959). This merge conflicted nowhere and auto-merged 22 files. A clean auto-merge of a keyed record is the case this project warns about, because both sides' edits apply and a removal can land invisibly. So .agents/model-matrix.md was checked line by line against origin/main rather than trusted: nothing main added is absent, and the only lines of main's that this branch replaces are the two counters its one ACTIVE row moves (ACTIVE 13->14, Total 381->382). The histogram sums to 382 and matches the printed Total; the engaged table holds 58 data rows and its prose says 58. FOLLOWING_AGENTS_PROTOCOL Following-Agents-Protocol: true AI-Assisted: true Assisted-by: AGENT:claude-opus-5-1m [claude-code]
…st GEMM refused
A served image on a CUDA build died in `vt: MatVec weight size mismatch at
deepseek_v4.cpp:504`. That guard is `Gemm`'s host-float fallback, and it named
no tensor, no layer and no geometry -- while the sibling keep-quant arm refused
the SAME wrong shape by name. The asymmetry is the defect: which tensor and
which layer were UNMEASURED precisely because the throw carried none of it, so
recovering the geometry needed an instrumented device run under a lease.
`MatVec` and `Gemm` now take the tensor name and the layer index, and the host
arm refuses in the keep-quant arm's own vocabulary:
deepseek-v4 host GEMM: weight size mismatch: tensor `comp_wgate` layer 3
want [N=6,K=8] = 48 elements, got 96 elements
The keep-quant arm gets the same two identifiers, so both arms now name the
same thing rather than one naming half of it.
THE LABELS ARE REQUIRED, NOT DEFAULTED. A defaulted label leaves a call site
anonymous, which is the defect this repairs; making them required means a call
site that forgets one is a `-Werror` build failure rather than a silent gap.
`check-tree-compiles` compiled 685 of 685 translation units in scope, which is
what proves no site was missed.
`VT_CHECK` evaluates its message inside the failure branch, so naming the site
costs nothing on the path that succeeds.
RED-FIRST. The new `test_deepseek_v4_forward` case enters through the
production entry `DeepseekV4ForwardHost` -- `Gemm` and `MatVec` are file-local
and no test can construct either -- and asserts content the anonymous string
does not contain: the tensor, the layer, both geometries and the actual element
count. One subcase breaks a not-layer-scoped weight (`lm_head`), the other
breaks `wq_a` on LAYER 2 and asserts the message says layer 2 and not layer 0,
so a hard-coded or first-layer index fails it. Both fail on the pre-repair tree
and pass on this one.
This is a DIAGNOSTIC change and that is the whole of it. It does not repair the
served-image mismatch, which stays owed by
ISSUE-LOCAL-01M29KEXRT2GCS6C53DT2S3SPX and #2411; it is what makes that
mismatch readable. The comments in `deepseek_v4.cpp` and
`deepseek_v4_weights.cpp` that cited the anonymous throw as the reason a
by-name loader refusal exists are corrected: those refusals still earn their
place, because they name WHAT IS MISSING and every mismatched tensor at once,
which a per-GEMM throw reports one at a time.
`tools/parity/dsv4v_matvec_named.sh` is the thor:gpu0 recipe that measures the
red, the green and the served image. It reads its own steps back, asserts the
two test binaries' md5s differ (`-Werror` means a mutant that fails to compile
leaves the OLD binary and a re-run prints SUCCESS), reads doctest verdicts off
`test cases:` rather than `Status:` (a non-matching `-tc` prints `test cases: 0`
AND `SUCCESS!`), and treats a PASSING red as a falsification rather than a pass.
FOLLOWING_AGENTS_PROTOCOL
Following-Agents-Protocol: true
AI-Assisted: true
Assisted-by: AGENT:claude-opus-5-1m [claude-code]
…sified The naming commit landed, and it made this row's spec say things that are no longer true. A record the tree falsifies is the failure AGENTS.md names, and a record edit rides in the pull request whose change made it stale, so it rides here. What changed, and nothing beyond it: - The blocker table's row 3 quoted the anonymous message as the LIVE blocker. The blocker IS still live -- the wrong shape is still thrown and a served image still does not complete -- but its message is now named, so the row says "LIVE, and now NAMED" rather than carrying a string the tree no longer emits. - "THE ASYMMETRY IS THE FINDING" described an asymmetry this branch closed. It now records that closing it is a DIAGNOSTIC and not the repair, which is the distinction the entry exists to hold. - "WHAT IS UNMEASURED" still says UNMEASURED, deliberately. The naming commit builds the instrument; it is not the measurement. The tensor, the layer and the geometry land in this entry only once a lease has actually printed them, and until that line exists the entry refuses to name a plausible candidate. - The stale cross-reference paragraph tracked this throw's LINE NUMBER, which went stale twice (`:413`, then `:504`). The comments now quote the MESSAGE and the function instead, so the paragraph records that the citations are line-free rather than correcting a number a third time. THE SPEC NAMES THE ISSUE BY DIRECTORY, NOT BY ID, and that is a rule rather than a preference. A first draft of this edit wrote the stable `ISSUE-LOCAL-...` ID into the entry, which sits under the spec's `## Owed`, and `scripts/check-agent-record.py` went from rc=0 to rc=1 on exactly that: a row-owned issue must not retain an owed reference, because an ID listed as owed is how a ROWLESS issue is tracked while a row-owned one is tracked by its directory. The checker caught it before it shipped and the entry now explains the constraint so the next editor does not repeat it. The issue file records the same split: one of its two owed halves is done, it STAYS OPEN for the other, and the geometry remains unmeasured. FOLLOWING_AGENTS_PROTOCOL Following-Agents-Protocol: true AI-Assisted: true Assisted-by: AGENT:claude-opus-5-1m [claude-code]
…on manifest gate The pr-size gate refused this pull request because a new 571-line checker carried no mutation evidence at the path the gate derives for it. The checker's 139 manifest cases live in a C++ loader test, which that gate cannot read. The new suite performs each failure instead of asserting it. It copies the checker and its three fixtures into a scratch tree, breaks one guarantee, and requires a named diagnostic and a counted number of disagreements. The counts are measured, not predicted. A bent vision count reds twice, because both manifests record it and the checker holds them to each other. One changed vision shape reds four times, because a rejected shape also leaves the records that the two digests cover. A config.json edit reds ten times including config_sha256, so the config and the manifests cannot drift apart. One case replaces urllib.request.urlopen with a function that fails the test, which proves the default mode reaches no network. check-pr-size.py gains a CREATION_MUTATIONS entry because this checker is absent at the range base. Without it the evidence run raised "absent at BASE and has no closed creation mutation", which reads an absent checker as a broken one. test_check_pr_size.py pins the entry and is the red-before evidence for that change. Measured: "Ran 28 tests" and OK at head. Under the disabled stub the suite reports "Ran 28 tests" then "FAILED (failures=19, errors=19)", and no case survives. The suite runs in both lanes, so a lane executes it: the SUITES array of scripts/agent-preflight.sh and the DeepSeek-V4 Vision step of .github/workflows/ci.yml. This change answers ISSUE-LOCAL-01M2BXFMFDNZQD41HY629KAGCC, which the row issue #2411 owns. FOLLOWING_AGENTS_PROTOCOL Following-Agents-Protocol: true AI-Assisted: true Assisted-by: AGENT:claude-opus-5-1m [claude-code]
…rward
A served image on a CUDA build died at the first GEMM of layer 0. The named
refusal this branch added says exactly what happened, and it is the whole
diagnosis:
vt: deepseek-v4 host GEMM: weight size mismatch: tensor `wq_a` layer 0
want [N=32,K=32] = 1024 elements, got 0 elements
`got 0` is the tell. This was never a wrong shape -- it is a weight that
reached NEITHER arm. `DeepseekV4Model::ForwardDevice` built its backend with
`gguf=nullptr` and bound only the EXL3 tower, so `ForwardComposeImpl` read
`kq_src = be.gguf != nullptr` as false and handed every layer `Lq = nullptr`.
Every `Gemm` then fell to the HOST-float arm, and on a GGUF load the host MLA
tower is empty BY DESIGN -- `deepseek_v4_weights.cpp` asserts `hl.wq_a.empty()`
in as many words, because the weight is meant to be consumed keep-quant. Layer
0's `wq_a` is simply the first GEMM the composition performs.
The sibling `DeepseekV4Model::Forward` has always dispatched on
`has_gguf_weights`; this entry never did. `git log -S'dev_be.gguf'` finds
nothing, so the device entry has not bound the tower since `dev_be` was
introduced in d7d1ee9.
WHY NO GATE CAUGHT IT. The registry sends the runner's default `gather_logits`
path to `ForwardDevice` unconditionally (deepseek_v4_registry.cpp:246), yet no
test has ever driven that entry with a GGUF tower: every `ForwardDevice` case
under `tests/` belongs to another architecture, and `test_cuda_deepseek_v4.cpp`'s
own `ForwardDevice ASSEMBLES` case sets `has_host_weights` with no GGUF tower at
all. A CPU build cannot reach the defect either, because `kDevicePending`
refuses first. So the combination "device entry + GGUF checkpoint" -- which is
what serving a real checkpoint DOES -- was unexecuted.
THIS BINDS THE TOWER RATHER THAN DELEGATING to `DeepseekV4ForwardGguf`, because
the four device op families are the point of this entry and delegating would
silently drop them. It is not a new combination: `Gemm`'s keep-quant arm already
retags the block views to the queue's device, and binding `gguf` sets
`dsa_dense` (:926), which turns the indexer and compressor arms off on every
layer -- exactly what the GGUF sibling already does, so no DSA arm sees a
half-bound backend.
NOT CLAIMED HERE: that an image now serves. That is a measurement, it needs a
CUDA lease because a CPU build cannot reach this code, and
`tools/parity/dsv4v_device_gguf_bind.sh` is the recipe that takes it. The driver
gates on the `image: ` LINE rather than on the suite's exit code, because
`test_deepseek_v4_mm_chat` also carries the separately-filed CPU-only-premise
assertion and so exits non-zero whatever the image does; it asserts the shipped
tree really carries this fix before believing any result, and it reports a
"stopped somewhere new" outcome as a named next blocker rather than as success.
FOLLOWING_AGENTS_PROTOCOL
Following-Agents-Protocol: true
AI-Assisted: true
Assisted-by: AGENT:claude-opus-5-1m [claude-code]
… cause
The spec promised these values would land "only when a lease has actually
printed them". A lease has printed them, so the entry that said UNMEASURED is
now false and is corrected.
MEASURED on thor:gpu0 (sm_110, CUDA 13.0.88, aarch64), rc job
04f39bcb-5636-4043-9cfc-8bdebd4862ec, from the served image:
vt: deepseek-v4 host GEMM: weight size mismatch: tensor `wq_a` layer 0
want [N=32,K=32] = 1024 elements, got 0 elements
Tensor `wq_a`, layer 0, N=K=32 -- the fixture's q_lora_rank by hidden_size.
Recovering this needed the naming commit: the anonymous throw carried none of
it, which is exactly why the values sat unmeasured through three blocker moves.
ROOT CAUSE, and `got 0` is what rules the shape hypothesis out. A wrong shape
gives a wrong COUNT; zero means the weight reached NEITHER arm. `ForwardDevice`
built its backend with `gguf=nullptr`, so `ForwardComposeImpl` read
`kq_src = be.gguf != nullptr` as false, handed every layer `Lq = nullptr`, and
every `Gemm` fell to the host arm -- whose MLA tower is empty BY DESIGN on a
GGUF load, because the weight is meant to be consumed keep-quant. Layer 0's
`wq_a` is merely the first GEMM the composition performs.
The record also says why no gate held it: the registry routes the default
`gather_logits` path to that entry unconditionally, while no test has ever
driven it with a GGUF tower, and a CPU build cannot reach it because
`kDevicePending` refuses first.
WHAT THIS COMMIT DOES NOT CLAIM. It does not say an image serves. The repair is
committed, and whether a served image COMPLETES or stops at a further blocker is
a separate measurement under a lease; the entry states that in those words so a
reader cannot mistake a diagnosis for an outcome. The issue stays OPEN for
exactly that reason.
FOLLOWING_AGENTS_PROTOCOL
Following-Agents-Protocol: true
AI-Assisted: true
Assisted-by: AGENT:claude-opus-5-1m [claude-code]
…built KV groups `vllm serve` aborted on every DeepSeek-V4 checkpoint. `test_serve_deepseek_v4_mm` died with SIGABRT inside `UnitaryKVCacheCoordinator` (`kv_cache_coordinator.cpp:350`, `!enable_caching || hash_block_size == block_size_`), so the server never answered and the suite reported `test cases: 2 | 0 passed | 2 failed`. Measured on a CPU Debug build at 5f86f9d with a temporary probe in that constructor: `groups=1 enable_caching=1 hash_block_size=256 block_size_=64 scheduler_block_size=256`, and `group[0] block_size=64` over 3 layers. The engine resolves 256 from DeepSeek-V4's `kv_block_size_floor` and builds the KV config at 256, but `MakeDeepseekV4KVCache` publishes the SWA group at a fixed 64, because upstream fixes that size by tensor sharing rather than by configuration (`vllm/v1/attention/backends/mla/sparse_swa.py:76-83`, `self.block_size = 64`). The engine then passed 256 as both the scheduler block size and the hash block size, because `scheduler.cpp` spelled `/*hash_block_size=*/block_size`. Upstream does two things this tree did not. It re-derives the cache block size from the groups that were actually built, `cache_config.block_size = min(g.kv_cache_spec.block_size for g in kv_cache_groups)` (`vllm/v1/engine/core.py:335-338` @ `e126687a9a`), and it then resolves that into a separate scheduler block size and hash block size and passes both to the Scheduler (`core.py:158-170`, `scheduler.py:76,268-270`), pairing the request block hasher with the hash size (`core.py:232`). Our `resolve_kv_cache_block_sizes` was already ported at `kv_cache_utils.cpp:640` and had no production caller, so both derivations were missing. This change adds `LoadedEngine::ResolveSchedulerBlockSizes` in the position upstream derives them, after the KV config exists and before the Scheduler is built, and threads the pair through `MakeScheduler` into `Scheduler` and `AsyncScheduler`. The new parameter defaults to 0, which is upstream's `None`, so every existing call site keeps its current meaning. The fix is the derivation, not the assertion. The assertion is upstream's own (`kv_cache_coordinator.py:516` @ the pin) and is untouched. RED: `test cases: 2 | 0 passed | 2 failed`, binary md5 8f1cf89ce2fd589b83a0019b9eb5b084. GREEN: `test cases: 2 | 2 passed | 0 failed`, binary md5 b7a5767653eb84b5d9cd3a50e8c08bff. Eleven neighbouring suites stay green: test_scheduler 48/48, test_scheduler_lpm 6/6, test_engine_core 6/6, test_kv_cache_coordinator 21/21, test_prefix_match_unit 8/8, test_prefix_cache_stats 12/12, test_loaded_engine_dense 30/30, test_dspark_draft_routing 7/7, test_deepseek_v4_mm_chat 8/8, test_deepseek_v4_mm_reach 20/20, test_deepseek_v4_mm_loader 17/17. A real Flash checkpoint publishes up to seven groups at block sizes 256, 64, 4 and 8, which no single hash granularity divides, so it still aborts in `HybridKVCacheCoordinator` on the deferred `BlockHashListWithBlockSize` path. That is pre-existing rather than introduced here, because the same configuration aborted before with 256 against 64 instead of the GCD, and it is recorded under `## Owed` in `.agents/specs/deepseek-v4-flash-vision.md`, including that the coordinator must refuse by name instead of asserting. This is also the engine-side half of the threading KV-PREFIX-MATCH-UNIT W3 deferred by name. `d957d94f6` landed `resolve_kv_cache_block_sizes` and recorded that "the scheduler still hardcodes hash_block_size = block_size, so the default/production path is byte-identical". That sentence is now false, so the records that carried it ride with this change: the `KV-PREFIX-MATCH-UNIT` rows in `.agents/engine-matrix.md` and `.agents/feature-matrix.md`, and the port map, gate and W-breakdown entries in `.agents/specs/prefix-match-unit.md`. Only the FINER-THAN-A-GROUP case is still deferred, because that is the one the `KV-BLOCK-POOL` align path gates (`block_pool.cpp:93,220`), along with the mamba partial-tail stop. The `## Our baseline` section of that spec is left as written: it is a snapshot of the tree at W1 time, and editing it would falsify history rather than repair staleness. The dense default stays byte-identical. One group whose block size equals the configured one resolves to exactly the pair the scheduler was already given, which is why the 48-case `test_scheduler` suite and the rest are unmoved. Closes ISSUE-LOCAL-01M2BXYS2R8ZNC68QYGHK4RVQH. FOLLOWING_AGENTS_PROTOCOL Following-Agents-Protocol: true AI-Assisted: true Assisted-by: AGENT:claude-opus-5-1m [claude-code]
…ware `test_deepseek_v4_mm_chat`'s image branch asserted that the served error names `W7-device`. That is `kDevicePending`, which `ForwardDevice` emits from `VT_CHECK(V4DeviceKernelsAvailable(), kDevicePending)` -- a predicate that is FALSE exactly when the V4 device kernels are ABSENT. On a build carrying them the refusal cannot fire, so the assertion was not merely unmet, it was UNSATISFIABLE: the case encoded the absent-kernel build as a premise (ISSUE-LOCAL-01M29KF3SNQCR8163840XSBK5S). The branch now selects on `vllm::deepseek_v4::V4DeviceKernelsAvailable()`, the same symbol `test_cuda_deepseek_v4.cpp` uses, so the two suites agree on what "this build carries the V4 device kernels" means. Without them the old expectation stands, because `kDevicePending` really is what that build emits. WHAT A CUDA BUILD ANSWERS IS MEASURED, NOT ASSUMED. On `thor:gpu0` (sm_110, CUDA 13.0.88), rc job `1b46515d-8caf-4c49-823e-efc7a1f3e3f4`, the request now travels the whole registered forward and stops at the MoE router's own named refusal: an image step routes on the vision bias `exp_probs_b_vl`, and the device router takes ONE bias for the call with no per-row selector, so it refuses BY NAME rather than routing image rows on the text bias. That device arm is W4-era code owed by #2411 W7-CUDA. IT ASSERTS THAT REFUSAL, not merely "not W7-device". A bare inequality would accept ANY failure, including a regression that stopped the request earlier -- which this row has already lived through twice, at the vision-residency refusal and then at the host GEMM's `wq_a` layer 0. When W7-CUDA lands the per-row bias, the request stops failing and takes the `served` branch instead. RECORDS. ISSUE-LOCAL-01M29KEXRT2GCS6C53DT2S3SPX is CLOSED with dated evidence: both halves it owed -- name the refusal, root-cause it -- are done and measured, and its blocker is gone from the served path. It does NOT close because an image serves; it closes because the anonymous MatVec refusal it describes is no longer live, while the new stop belongs to a different owner. The spec's blocker table records that move: row 3 CLOSED, row 4 (the MoE vision-bias device router) LIVE, and the count of measured blocker moves goes from three to four. The driver gains a `TARBALL` selector so a later wave need not overwrite an earlier wave's staged tree -- overwriting is how a run ends up measuring a tree nobody can identify afterwards -- and a `mm_chat_suite_green` step that gates on `0 failed` from doctest's COUNTED line, since this suite read `8 | 7 passed | 1 failed` before this change. Its header no longer claims the suite "exits non-zero whatever happens", which this commit falsifies. FOLLOWING_AGENTS_PROTOCOL Following-Agents-Protocol: true AI-Assisted: true Assisted-by: AGENT:claude-opus-5-1m [claude-code]
…n CUDA The device-aware image expectation is measured green on a build that carries the V4 device kernels, so the issue it repairs closes with that evidence rather than on the strength of the change having been written. thor:gpu0 (sm_110, CUDA 13.0.88), output dir `bind-20260912-235206`, measuring the exact tree of 5ffc29b: `test_deepseek_v4_mm_chat` reads `test cases: 8 | 8 passed | 0 failed | 0 skipped` with `mm_chat_suite_green RC=0`. The red it replaces was measured on the SAME box and the SAME suite at `8 | 7 passed | 1 failed`, the one failure being the unsatisfiable `W7-device` assertion. Verdicts come off doctest's COUNTED line, never `Status:`. Nothing regressed beside it: `test_deepseek_v4_forward` 7 of 7 and `test_deepseek_v4_mm_reach` 20 of 20 under `VT_CPU_QUANT_REPACK=0`. THE DRIVER'S BLOCKER-PRINT WAS REPORTING THE WRONG LINE, and that is repaired here because a recipe that misnames a cause is worse than one that prints nothing. `grep 'image: '` also matches the PNG and data-URI residual INFO lines, which read `chat image: ...` and are NOT the engine's stop: on the run that prompted this, the bare pattern printed three container-decode residuals while the actual refusal sat ~70 lines further down. It now greps the case's own `MESSAGE: image: ` line and `engine-fatal`. No `step` decision changes, so the gate this run passed is the same gate; only what a reader is handed changes. The engine's stop is quoted from the log rather than inferred from the suite passing: `vt: deepseek-v4 MoE: this step carries image rows, which route on the vision bias exp_probs_b_vl, and the device router takes one bias for the whole call with no per-row selector`, at deepseek_v4.cpp:437. That is W4-era code with its own named refusal, owed by #2411 W7-CUDA, and it is the live blocker for a served image -- not a regression from this branch. FOLLOWING_AGENTS_PROTOCOL Following-Agents-Protocol: true AI-Assisted: true Assisted-by: AGENT:claude-opus-5-1m [claude-code]
… fixed Owed entry Two statements in this spec were falsified by this branch's own measurements, and a record the tree falsifies is the failure AGENTS.md names. FIRST, the entry said "WHAT REMAINS OWED IS THE OUTCOME ... never a working image". The outcome has since been measured twice under the lease (`bind-20260912-232118`, `bind-20260912-235206`): the `wq_a` refusal is GONE (`image_past_wq_a` RC=0), the request travels the whole registered forward, and it STOPS at blocker 4 in this entry's own table -- the MoE vision-bias device router, W4-era code refusing by name. The entry now says that, and says plainly that it claims a root cause, a repair and a MOVED blocker, never a served image. What a served image still needs is the device MoE arm, which belongs to #2411 W7-CUDA rather than to this entry. SECOND, the `## Owed` bullet describing `test_deepseek_v4_mm_chat`'s CPU-only premise was still written as an open defect after this branch repaired it. It now records the repair and the two-way measurement on the same box and suite: `8 | 7 passed | 1 failed` before, `8 | 8 passed | 0 failed` after. Leaving a fixed item under Owed is how an intake grows without an exit, which this protocol calls out by name. Neither edit claims an image serves, because it does not. FOLLOWING_AGENTS_PROTOCOL Following-Agents-Protocol: true AI-Assisted: true Assisted-by: AGENT:claude-opus-5-1m [claude-code]
…-Vision row FOLLOWING_AGENTS_PROTOCOL Following-Agents-Protocol: true AI-Assisted: true Assisted-by: AGENT:claude-opus-5-1m [claude-code]
…d into the DeepSeek-V4-Flash-Vision row FOLLOWING_AGENTS_PROTOCOL Following-Agents-Protocol: true AI-Assisted: true Assisted-by: AGENT:claude-opus-5-1m [claude-code]
FOLLOWING_AGENTS_PROTOCOL Following-Agents-Protocol: true AI-Assisted: true Assisted-by: AGENT:claude-opus-5-1m [claude-code]
…ration Main gained 50 commits while the three CI repairs were built. This merge conflicted nowhere and auto-merged 83 files. A clean auto-merge of a keyed record is the case this project warns about, so .agents/model-matrix.md was checked against origin/main line by line rather than trusted: nothing main added is absent, and the only lines of main's that this branch replaces are the two counters its one ACTIVE row moves (ACTIVE 13->14, Total 381->382). The histogram sums to 382 and matches the printed Total. The three repairs this integration carries were each re-verified present after the merge, by their load-bearing markers rather than by the merge reporting success: the ForwardDevice gguf binding, resolve_kv_cache_block_sizes reached from three sites, the named host GEMM refusal, the manifest mutation suite and its CREATION_MUTATIONS entry. FOLLOWING_AGENTS_PROTOCOL Following-Agents-Protocol: true AI-Assisted: true Assisted-by: AGENT:claude-opus-5-1m [claude-code]
…prediction the tree falsifies
`.agents/specs/deepseek-v4-flash-vision.md` `## Owed` said a real DeepSeek-V4-Flash checkpoint ABORTS in `HybridKVCacheCoordinator` on its `block_size == hash_block_size` assert, and that the coordinator must therefore learn to refuse by name. That was a prediction written from reading the code, and a probe through the production loader disproves it. This commit replaces the prediction with the measurement and adds the probe that made it.
WHAT WAS MEASURED. A GGUF fixture carrying real `compress_ratios` {4, 128, 0} driven through `LoadedEngine::FromModelDir` on its DEFAULT configuration. The topology half of the prediction holds exactly: `MakeDeepseekV4KVCache` publishes seven groups at block sizes {256, 256, 256, 64, 4, 4, 8}, and `resolve_kv_cache_block_sizes` resolves scheduler 256 (the LCM) and hash 4 (the GCD), which the engine has derived since `c9129fb7b`.
WHAT DOES NOT HOLD. The engine never reaches the coordinator. `ApplyCacheDType` runs while `kv_cfg_` is being initialized, which precedes `scheduler_block_size_` and `scheduler_` in the `LoadedEngine` constructor's initializer list, and `RetypeAttentionSpec` refuses any `MLAAttentionSpec` BY NAME at `src/vllm/v1/kv_cache_interface.cpp:398` — the fp8_ds_mla page formula landed with no store and no read (#2455, owed to KV-DSV4-MULTICACHE W8). The compressed-latent groups are exactly the groups a non-zero `compress_ratio` adds, so the ratios that build the multi-group topology are also what trips that guard. A named refusal is a message rather than an abort, so the serve-time outcome today is already the acceptable one, for a different reason than the record claimed.
WHY THE ASSERT IS UNREACHABLE RATHER THAN MERELY UNTRIGGERED, which is the load-bearing part. DeepSeek-V4 is the only architecture in this tree that publishes groups with DIFFERING block sizes; every other multi-group registry — `glm5_next`, `kimi_linear`, `nemotron_h`, `qwen4_exp`, `qwen3_5_common` — hands the same `block_size` variable to every group it publishes. So no production entry point can reach `kv_cache_coordinator.cpp:386` or `block_pool.cpp:93,220` today.
WHAT IS DELIBERATELY NOT HERE. The `BlockHashListWithBlockSize` port. It would be a converting view that nothing can reach, which is dead code with paperwork attached, and this row has already paid for that shape more than once. #2455 / W8 is what makes the path reachable, and the port is owed behind it. So that W8 does not have to rediscover the shape, `## Owed` now records it with anchors, all @ `e126687a9a`: `kv_cache_utils.py:678-770` (LCM/GCD, already ported and wired), `:2358-2464` (`BlockHashListWithBlockSize` / `resolve_block_hashes`, NOT ported — a coarse block's hash is the last fine hash inside it), `kv_cache_coordinator.py:608-613` (upstream asserts DIVISIBILITY only, so our equality at `kv_cache_coordinator.cpp:386` is a local deferral marker and not upstream's), and the four call sites to mirror.
THE TEST ASSERTS TODAY'S BEHAVIOUR, not a future one. `test_deepseek_v4_multigroup_kv` pins the seven-group topology and the (256, 4) resolution, then requires the load to be REFUSED naming `fp8_ds_mla` and #2455, and NOT naming `hash_block_size`. Its second case is a tripwire: when that refusal stops firing, W8 has landed and the coordinator assert becomes reachable, which is stated in the case itself.
EVIDENCE. RED (the prediction's fixture, asserting a successful load): `test cases: 2 | 1 passed | 1 failed`, binary md5 `9cc3b4fd8338c6483d17bf481b3459df`, the failure being the verbatim fp8_ds_mla refusal quoted above. GREEN (asserting the refusal): `test cases: 2 | 2 passed | 0 failed`, `assertions: 16 | 16 passed`, binary md5 `479f344f97ba81ff2d3557043f96c2d0` — a CHANGED binary, so the verdict is not a stale artifact.
THE SHARED FIXTURE EDIT IS BYTE-NEUTRAL. `BuildDeepseek4Gguf` gained a defaulted `compress_ratios` parameter and the per-layer DSA compressor/indexer tensors that real ratios require; empty (the default) writes the all-zero array every existing caller relied on. Its five consumers are unmoved: `test_serve_deepseek_v4_mm` 2/2, `test_deepseek_v4_mm_reach` 20/20, `test_deepseek_v4_mm_loader` 17/17, `test_deepseek_v4_mm_chat` 8/8, `test_deepseek_v4_gguf_load` 19/19.
No file under `src/` or `include/` is touched: the upstream assertion is exactly where it was.
Records: ISSUE-LOCAL-01M2C0JBH6CNPYGQXNF0A1WG0H is corrected to state the measurement and stays OPEN, because a multi-group DeepSeek-V4 still cannot prefix-cache.
FOLLOWING_AGENTS_PROTOCOL
Following-Agents-Protocol: true
AI-Assisted: true
Assisted-by: AGENT:claude-opus-5-1m [claude-code]
…evice MoE router
`DispRoute` refuses any step carrying image rows on the two DEVICE router arms,
because `MoeDeviceKernels::route` and `route_ip` each take ONE bias pointer for
the whole call and have no per-row selector. That refusal is correct and
deliberate: routing an image row on the text bias would be fluent and wrong. What
is missing is the selector, and this section scopes it before any code is
written.
It is the LAST KNOWN BLOCKER to serving an image on CUDA, and that is a measured
position rather than an expectation. `ForwardDevice` builds
`V4Backend dev_be{/*device=*/true, ...}` and the registry sends the runner's
default gather-logits path there for every request on this architecture, so every
served request takes the `be.device` arm. With the vision-residency and
keep-quant-binding repairs above it in, a served image travels the whole
registered forward and stops exactly here.
The section records the ONE predicate both the refusal and the route must share,
transcribed from the host arm rather than restated, because a refusal whose
predicate differs from its routing condition is how a silent wrong answer
happens and this row has already met that defect class. It cites vLLM as the
primary anchor for the arithmetic of each arm and names the point where vLLM
stops: vLLM takes one `e_score_correction_bias` and implements no vision bias, so
the per-row selection between two biases is the secondary oracles' ground. It
also states plainly that a CPU build CANNOT gate this work, since `ForwardDevice`
refuses first at `kDevicePending`, so no CPU green is device coverage.
The design section sits OUTSIDE `## Owed` deliberately. That heading spans to
`## Now`, and `check-agent-record.py` reds a row-owned issue ID mentioned in a
spec's `## Owed`, so the new local issue is named here and nowhere in that range.
ISSUE-LOCAL-01M2C26CSZWB7WVRS5H7YPW4S8
FOLLOWING_AGENTS_PROTOCOL
Following-Agents-Protocol: true
AI-Assisted: true
Assisted-by: AGENT:claude-opus-5-1m [claude-code]
…-Flash-Vision row The issue and the spec Owed entry both asserted that a real Flash checkpoint aborts in HybridKVCacheCoordinator. Measurement disproved it: the checkpoint never reaches the coordinator, because ApplyCacheDType runs during kv_cfg_ init and RetypeAttentionSpec refuses any MLAAttentionSpec by name at kv_cache_interface.cpp:398. The coordinator equality assert is latent, not live, and what a real checkpoint needs next is #2455 / KV-DSV4-MULTICACHE W8. FOLLOWING_AGENTS_PROTOCOL Following-Agents-Protocol: true AI-Assisted: true Assisted-by: AGENT:claude-opus-5-1m [claude-code]
…gration Three further commits landed on main while this row was gating. This merge conflicted nowhere and auto-merged 8 files. .agents/model-matrix.md auto-merged, which is the case this project warns about, so it was checked against origin/main line by line rather than trusted: nothing main added is absent, and the only lines of main's this branch replaces are the two counters its one ACTIVE row moves (ACTIVE 13->14, Total 381->382). The histogram sums to 382 and matches the printed Total. The repairs this branch carries were re-verified present after the merge by their load-bearing markers rather than by the merge reporting success: the ForwardDevice gguf binding, resolve_kv_cache_block_sizes reached from three sites, the named host GEMM refusal, and the manifest mutation suite. FOLLOWING_AGENTS_PROTOCOL Following-Agents-Protocol: true AI-Assisted: true Assisted-by: AGENT:claude-opus-5-1m [claude-code]
… selector
`DispRoute` refused every step carrying image rows on the two DEVICE router arms,
because `MoeDeviceKernels::route` and `route_ip` each took ONE bias pointer for
the whole call. That refusal was correct: routing an image row on the text bias
would have been fluent and wrong. What was missing was the selector, and this
adds it rather than widening the refusal.
THIS IS THE LAST KNOWN BLOCKER TO SERVING AN IMAGE ON CUDA, and that is measured
rather than expected. `DeepseekV4Model::ForwardDevice` builds
`V4Backend dev_be{/*device=*/true, ...}` and the registry sends the runner's
default gather-logits path there for every request on this architecture, so every
served request takes the `be.device` arm. With the vision-residency and
keep-quant-binding repairs above it in, a served image travels the whole
registered forward and stops exactly here.
ONE PREDICATE, TRANSCRIBED RATHER THAN RE-DERIVED. A refusal whose predicate
differs from its routing condition is how a silent wrong answer happens, and this
row has already met that defect class. Both CUDA kernels now compute, per row,
`media = has_vbias && is_media && is_media[t] != 0`, then take
`row_bias = media ? vision_bias : bias`, `row_has_bias = media ? 1 : has_bias`
and `row_is_hash = is_hash && !media`. That is `SqrtSoftplusRouteTopk`'s rule
verbatim: an image row takes `exp_probs_b_vl` and the learned top-k, and on a
HASH layer that REPLACES the tid2eid route for that row alone while a text row in
the same step still hashes. Weights are still gathered from the UNBIASED scores.
BOTH kernels carry it, not just the default one. `RouteWarpKernel` is shipped-on
and `RouteKernel` is its A/B baseline, and the two must stay bit-identical on a
media step as well as a text one. All 32 lanes of a warp serve one row, so they
agree on `media` with no divergence.
THE REFUSAL WAS ANSWERED, NOT DELETED. The genuinely-unavailable case is a
DIFFERENT predicate and keeps its own refusal one level up in `MoeBlock`: a step
carrying image rows whose LAYER has no `exp_probs_b_vl` is refused by name, with
the width check beside it. That one is host-side and arm-independent, so a text
checkpoint asked to route an image is still refused on every arm.
A TEXT STEP IS BYTE-IDENTICAL. An empty mask or an absent vision bias means no
row is media -- the host arm's own `any_media` rule -- and `RouteLaunch` pads the
two new buffers exactly as it already pads `bias`, `in_tokens` and `hashtab`, so
no kernel argument is ever a dangling pointer.
The two resident single-token decode arms pass the no-media values. They refuse
an out-of-vocabulary identifier before the router, so no image row can reach
them; widening that refusal is separate work and is NOT done here. The decode
graph's guard keeps its `embed`-bound reason and loses the one the tree has now
falsified.
WHAT IS PROVEN, AND ON WHAT. On x86-64 CPU: build green under -Werror, all three
binaries' md5 CHANGED, `test_deepseek_v4_moe` 14/14 (731 assertions),
`test_deepseek_v4_mm_reach` 20/20 (178), `test_deepseek_v4_mm_chat` 8/8 (650).
That is NOT device coverage and is not offered as any: a CPU build cannot reach
this code, because `ForwardDevice` refuses first at
`VT_CHECK(V4DeviceKernelsAvailable(), kDevicePending)` -- observed in the CPU
`mm_chat` run, which stops exactly there.
THE DEVICE ARM IS MEASURED, on `thor:gpu0` (NVIDIA Thor, compute_cap 11.0,
driver 595.78, CUDA 13.0.88, aarch64), rc job
`bc9c74ce-e9f6-47e1-bde0-bb0939ad0ed8`. There is no local nvcc, so that run is
the FIRST compile of `cuda_deepseek_v4.cu` with the selector: 41 `.cu.o`, the V4
objects `sm_110`, linked against `libcudart.so.13` and `libcublasLt.so.13`,
binaries built by that run (`test_cuda_deepseek_v4` md5
`12b23adb4b6b4741bad7c19626f70211`, `test_deepseek_v4_mm_chat` md5
`625faa37ecdfa3716c38a3ba8ddec628`).
- the focused case `W7-CUDA per-row vision bias`: `test cases: 1 | 1 passed`,
102 assertions. The matched count is read explicitly, because a non-matching
`-tc` prints `test cases: 0` AND `Status: SUCCESS!`;
- the whole CUDA V4 suite: `31 | 31 passed | 0 failed | 0 skipped`, 90193
assertions -- zero skipped, so no case silently sat out;
- `test_deepseek_v4_mm_chat`: 8/8, 650 assertions, and the served image reads
`image: served`, with `VT_CPU_QUANT_REPACK` ON and OFF alike, so the answer
is not an artefact of the aarch64 i8mm repack path.
AN IMAGE SERVES ON CUDA. The limits are stated rather than left to be assumed:
the fixture is the SYNTHETIC lang GGUF plus `deepseek4v` mmproj built by the
suite and driven through the production entry point, NOT the released
checkpoint, so no real-weight image answer exists. AND THE REACHABILITY MUTATION
WAS NOT RUN: deleting the `DispRoute` pass-through and confirming the focused
gate reds is what would prove the green rests on the production call site rather
than on a class that merely works. The spec records both under `## Owed`.
`test_deepseek_v4_mm_chat`'s device branch is the gate and was RED-first: it
asserts the served-image error is EMPTY, not merely "not the old refusal", so a
regression that stopped the request earlier cannot pass it.
The records this result falsified are repaired in the same commit: the two
`## Owed` device-router bullets, the blocker table (row 4 CLOSED, and a row 5
that says "no further blocker is KNOWN on the synthetic fixture", never "none
exists"), `## Now`, and the FEATURES.md sentence that read "NO IMAGE HAS BEEN
SERVED END TO END".
ISSUE-LOCAL-01M2C26CSZWB7WVRS5H7YPW4S8
FOLLOWING_AGENTS_PROTOCOL
Following-Agents-Protocol: true
AI-Assisted: true
Assisted-by: AGENT:claude-opus-5-1m [claude-code]
…sion row An image now serves on CUDA. On thor:gpu0 (NVIDIA Thor, compute_cap 11.0, driver 595.78, CUDA 13.0.88, aarch64), rc job bc9c74ce, a two-image chat request travels create_chat_completion -> the MM seam -> AsyncLLM -> Scheduler -> GPUModelRunner -> ModelRegistry::Forward and returns `image: served`, with VT_CPU_QUANT_REPACK on and off alike, so it is not the aarch64 repack path. Focused case 1/1 with 102 assertions; the CUDA V4 suite 31/31 with 90193 assertions and zero skips; mm_chat 8/8. The W4 refusal was ANSWERED, not deleted. Both device routers now carry the per-row selector, so DispRoute holds no VT_CHECK at all, while the genuinely unavailable case keeps its own by-name refusal one level up in MoeBlock: a step with image rows on a layer lacking `exp_probs_b_vl` is refused there, host-side and arm-independent, on every arm. Two limits are recorded in the spec's `## Owed` rather than papered over: the fixture is the synthetic lang GGUF plus deepseek4v mmproj, NOT the released checkpoint, so no real-weight image answer exists; and the reachability mutation has not been run, so nothing yet proves this green rests on the production call site rather than on a class that merely works. A third is owed too: the vLLM router was read at 5559679229, the prior parity pin, not the active e126687a9a. FOLLOWING_AGENTS_PROTOCOL Following-Agents-Protocol: true AI-Assisted: true Assisted-by: AGENT:claude-opus-5-1m [claude-code]
…ration One commit landed on main during the previous integration, a QWEN4-EXP MoE adapter residency fix unrelated to this row. It auto-merged across 5 files with no conflict. .agents/model-matrix.md auto-merged again, so it was checked rather than trusted: both of this row's entries are present exactly once, the state histogram sums to 382 and matches the printed Total, and the five repairs this branch carries were re-verified by their load-bearing markers after the merge -- the ForwardDevice gguf binding, resolve_kv_cache_block_sizes reached from three sites, the named host GEMM refusal, the per-row vision bias selector, and the manifest mutation suite. FOLLOWING_AGENTS_PROTOCOL Following-Agents-Protocol: true AI-Assisted: true Assisted-by: AGENT:claude-opus-5-1m [claude-code]
…ration One QWEN4-EXP spec commit landed on main during the gate run; it auto-merged across 2 record files with no conflict and touches nothing this row builds. .agents/model-matrix.md was checked rather than trusted, as on every previous integration: both of this row's entries appear exactly once, the state histogram sums to 382 and matches the printed Total, and the five repairs were re-verified present by their load-bearing markers after the merge. FOLLOWING_AGENTS_PROTOCOL Following-Agents-Protocol: true AI-Assisted: true Assisted-by: AGENT:claude-opus-5-1m [claude-code]
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
The vision arm of DeepSeek-V4-Flash-Vision now loads, runs on a CUDA device and
matches the pinned oracle. It does not yet serve an image end to end, and this
change says where it stops.
What this lands
The tower runs on the device and matches the oracle. On
thor:gpu0(sm_110), all four
lead_padrungs run onprovider: cudaand compare directlyagainst llama.cpp
b10766: the four sentinel kinds are byte-exact (max_abs 0.0), the block permutation is the identity for 100/100 image rows, cells meanrelative L2 is 2.884% at cosine 0.99939, and vit mean relative L2 is 1.872%.
The device arm is closer to the oracle than our own CPU arm (3.83% / 0.99899).
This was red first: every rung aborted on
DeepSeek-V4 vision queue and weights must share one devicewhile the same binary's CPU arm reproduced the W6 blockbyte for byte, which is what makes it a device result rather than a build
difference.
The staging fix is gated on a CPU host.
EnsureResidentstages the tower tothe queue's device. A record previously claimed a per-block staging miss could
only be caught on a leased device; that was false. A hand-built
kXPUqueue anda counting backend reach
EnsureResidentthrough the productionForward, so amiss on any block index now reds on a CPU box with no GPU: 23 staged copies at
depth 2, 15 if block 1 is skipped, per-block slope 8.
The official safetensors vision loader. 267 vision tensors in shard 1 were
never read and — because the dense name-map pass has no leftover refusal — never
even counted. They are now loaded and accounted for on both the dense and EXL3
arms, reached from
LoadDeepseekV4ForCausalLM's safetensors branch. Weights arecopied out of the mmap rather than borrowed: safetensors offsets carry no
alignment guarantee, and this tree's own fixture writer forces an odd payload
base, so a borrowed bf16 view can start at an address no bf16 access may assume.
The tensor map is pinned to the released artifact. A 571-line offline
checker holds the loader to the measured shard-1 header — 72,633 tensors, 267 of
them vision, 932,786,176 payload bytes — with three recorded fixtures. Mutating
the recorded vision count 267 -> 266 exits 1 naming both disagreements.
The parity harness can now fail. This is the largest correction here. The
W6/W7 comparator had no bound and exactly one non-zero exit: a drift from 3.83%
to 12% produced rc 0, a well-formed report and a DONE banner. Four drivers
recorded
tee's exit status instead of the command's, wrote asteps.txtnothing read back, and treated a step that never ran as a step that passed. Five
rounds of repair and five independent reviews later, every judging key, every
stage and the separability precondition must be declared; a non-finite statistic
is a failure rather than a silently disabled bound; the permutation margin is
derived per run from the bf16 grid spacing rather than a guessed constant; and a
new 48-case suite reds when any of it is reverted.
Scope
91 commits, spanning W1 through W7 of this row across several sessions. Roughly
half the volume is recorded data rather than code:
goldens.jsonis 16,387lines, the shard-1 header manifest 1,755, the GGUF name manifest 1,371.
Verification
Release build,
VLLM_CPP_CUDA=OFFon this host. Family gate 26/26 with zero NotRun and every binary confirmed present; the W6 comparator suite 48/48; preflight
clean with no failing gate. Device numbers come from
rcleases onthor:gpu0(jobs
14908980,665b2427,c472faab);test_cuda_deepseek_v4ran 29 casesand 90,082 assertions with zero skips on sm_110, and exits 77 on a CPU host, so
its green is a device green.
What this does NOT do
#2411stays open. A served image on a CUDA build still fails, now at ananonymous
vt: MatVec weight size mismatchinGemm's host-float fallback --device-specific only in reachability, since a CPU build refuses earlier at
kDevicePending. The blocker moved three times under measurement (fp8_ds_mla ->vision residency, closed -> MatVec, live), and each move was visible only
because the previous one was genuinely repaired.
The real 156.287 GiB safetensors payload has never been read. Names, shapes,
dtypes and counts are pinned to the measured header; values are proven only
against synthetic fixtures. The W6/W7 jobs have not been re-run under the
repaired harness, which needs a lease, so no real leg has yet reported its own
margin or rounding scale.
The 4.9% / 0.998 bound should not be quoted as independent evidence: given
condition (2), the triangle inequality already forces it. The conditions doing
real work are sentinels-exact-with-identity-permutation and the f32 arm inside
the oracle's own floor.
Eight local issues stay open and are named in the records: the MatVec blocker;
test_deepseek_v4_mm_chat's image branch encoding a CPU-only premise that cannever hold on CUDA; 8 of 20
mm_reachcases failing on aarch64 from the i8mmquant repack, proven by a same-binary A/B;
test_serve_deepseek_v4_mmtimingout at 1800 s with no output and no guessed cause; three unproven W6 harness
weaknesses; moved spec line anchors in the W7-CUDA driver; three malformed
bounds inputs that crash instead of reaching the ERROR exit; and a stray
NN - <text>line inside a ctest FAILED block read as a test name.FOLLOWING_AGENTS_PROTOCOL
Following-Agents-Protocol: true
AI-Assisted: true
Assisted-by: AGENT:claude-opus-5-1m [claude-code]