feat(MODEL-MM-QWEN4-EXP): stage a large pageable H2D through a bounded pinned ring on ROCm by localai-org-maint-bot · Pull Request #3180 · mudler/vllm.cpp

localai-org-maint-bot · 2026-09-13T06:49:28Z

The 67.56 GiB Qwen3.8-Flash-Next UD-IQ1_S loads on gfx1151 with zero op refusals
and then never produces a token. .agents/specs/rocm-host-residency-after-upload.md
§6a measured it twice: killed at a 1200 s deadline, compute thread uninterruptible
in svm_range_set_attr for about three quarters of its wchan samples, device
memory stuck at 29.69 GiB. Releasing the spent source pages and stopping the
prefault both landed and neither moved it. That spec's §7 fourth risk names this
change as what is owed next, and ISSUE-LOCAL-01M2BZ5QK4XRETK48CXKSHKRDW owns it.

RocmBackend::Copy was one hipMemcpyAsync handed a multi-GiB pageable,
file-backed, CIFS-backed source. To DMA out of a pageable range the ROCr runtime
has to make all of it resident and describe it to the KFD, which is where that
wchan comes from. This stages such a copy through four pinned 64 MiB buffers
with an event each -- wait on the slot, fill it, upload from it, record, advance
modulo four -- so host residency for the transfer is 256 MiB for a model of any
size. That is llama.cpp's own ring at the recorded pin 10bf611e5,
src/llama-model-loader.cpp:1440 (n_buffers = 4), :1449 (64 MiB),
:1496-1516 and :1591-1642. The spec states the one place the oracle and we
differ rather than glossing it: the oracle takes that ring only when it is NOT
using mmap, so what is ported is the shape, not the call.

The reach, enumerated before the code was written

Backend::Copy is a shared seam and this row has already shipped a seam change
whose blast radius nobody wrote down. So §3b of the spec enumerates it first.
ONE override is touched -- src/vt/rocm/rocm_backend.hip -- plus a new HIP-free
header. CUDA, CPU, Metal, Vulkan, XPU and Tenstorrent are not edited and their
Copy is byte-identical. Within ROCm the staged path needs five terms at once:
at least one chunk of bytes, a device destination, an unregistered host source, a
stream that is not capturing, and a ring that allocated. A 4 KiB norm weight, a
readback, a device-to-device copy, an already-pinned or managed source, and
anything inside a graph capture all take the previous single call unchanged.

What that reaches is wider than the previous fix's: it does not require
mmap_fd, so on an AMD board it fires for every safetensors model and the EXL3
trellis uploads as well as for the five GGUF keep-quant families. Only Qwen4-Exp
was run, and the spec records that rather than implying coverage.

Red, green, and the mutations

Measured on strix:gpu0 (gfx1151, ROCm 7.2.4) inside an rc lease, job
e8bf3b66-eb80-4659-8e6e-48167eb6bf60. Every selector names its binary and its
counts, because this row has twice shipped a selector that matched nothing and
printed Status: SUCCESS!.

At the test commit e1bf7fd1d, test_backend_cross_device -tc=*pinned bounce*
reported 1 case / 8 assertions, 5 failed, FAILURE, staged=0 chunks=0 ring_bytes=0. At 81f91b600 the same selector on the same board reported 1 / 8
green with staged=1 direct=2 chunks=3 max_chunk=67108864 ring_bytes=268435456.

binary	selector	cases	assertions
`test_rocm_pinned_h2d`	whole binary	10	2007
`test_rocm_pinned_h2d`	`-tc=cheap terms`	1	1923
`test_backend_cross_device`	`-tc=pinned bounce`	1	8
`test_backend_cross_device`	`-tc=pinned bounce`, `VT_ROCM_MANAGED_ALLOC=1`	1	4
`test_backend_cross_device`	whole binary	61	84841
`test_backend_cross_device`	`-tc=DSA`	2	273

(The unit binary's 10 / 2007 is post-round-2; see the sections below. It was
9 / 84 when the implementation first landed and 10 / 1527 after the first
repair. The counts were re-measured on strix:gpu0 at 92d3305cd, rc job
ecf7b0b6-6e6f-4123-b6d2-50d5178a32dd, every selector printing its case and
assertion counts and both cross-device selectors printing
pinned H2D case ran on a ROCm board: 1.)

The baseline at 98e2cd7da is 60 / 84833 and 2 / 273, so the suite grew by
exactly this one case and its eight assertions and -tc=*DSA* is unmoved.

Deleting the production call site from RocmBackend::Copy fails the device case
with staged=0 direct=3; widening the chunk to the whole buffer fails both
binaries. Both binaries were proven changed by md5 on every arm, and the rebuild
after them reproduced the clean implementation hashes exactly, which is what
proves the restoration rather than git status. Eight further mutations of the
pure header were run off-board and each convicted.

The model gate, which is the deliverable

rc job 672093bc-932b-4e54-b319-e15529f70256, box idle and exclusively leased,
VT_ROCM_MANAGED_ALLOC unset, artifact asserted HIP-linked before it was timed.
Arm C is arm A with the three shards copied to local disk first, which separates
the CIFS confound §6a left open.

	A: ring ON, CIFS	B: ring OFF, CIFS, SAME BINARY	C: ring ON, local
token	3 x 32, `finish_reason=length`	none, killed at 1200 s	3 x 32
wall	771 s	1218 s	62 s
`[vt load] weights`	61.187 s	35.909 s	8.770 s
peak `VmHWM`	20,714,504 kB	27,076,580 kB	26,920,604 kB
peak `RssFile`	14,516,792 kB	21,093,296 kB	20,728,476 kB
peak device memory	77,271,658,496 B	31,873,912,832 B	77,271,609,344 B
`wchan`	115 `folio_wait_bit_common`, 0 `svm_range_set_attr`	135 `svm_range_set_attr` of 197	nothing in D state

The off arm reproduces the recorded wedge to within noise on every axis, minutes
apart, from one binary, so the ring is the cause rather than a coincidence of the
day. Device memory is the tell: arm B stops at the same 29.69 GiB plateau §6a
recorded, arm A and arm C reach 71.96 GiB, which is the whole checkpoint.

Decode: 5.0-5.3 tok/s -- nine samples across six independent process
launches, 5.002 to 5.291, 5.8% max-to-min. This is quoted as a RANGE because the
first reading here was "5.291, 5.273, 5.274, 5.275, spread 0.34%" off two
launches, and that precision does not reproduce. Each arm's first generation (695.871 s on CIFS, 38.438 s locally) carries the
one-time lazy staging of the whole 72 GiB checkpoint and is not a decode number.
Arm C shows that 657 of arm A's 696 s were CIFS read wait, and that the ring
moves the checkpoint onto the board in about 38 s. No TTFT is quoted, because
vllm-cli in blocking mode reports whole-generation seconds. No throughput
comparison is claimed: no oracle ran this workload.

The repair wave: what a fresh review found, and what it cost

A fresh review returned FAIL. Four findings, all addressed here.

The ring was allocated before the decision that would use it.
StagedCopy evaluated in.ring_available = EnsureRing(chunk) BEFORE
ShouldStageH2D, so the first copy of 64 MiB or more on a non-capturing stream
built 256 MiB of pinned host memory and four events whatever the pointers turned
out to be -- including a D2D copy, an already-pinned source, or a managed
destination that then took the direct path. Measured, not inferred:
VT_ROCM_MANAGED_ALLOC=1 on gfx1151 printed
staged=0 direct=3 chunks=0 max_chunk=0 ring_bytes=268435456 and the case still
said SUCCESS!, because the managed arm asserted nothing about ring_bytes.
The instrument printed the defect and nothing read it. That is not a free
256 MiB on the boards that show it: .agents/environment.md measures gfx1151's
managed ceiling as bounded by HOST RAM, 27 GiB against 29.3 GiB available, so
pinned host memory comes out of the resource that decides whether a checkpoint
fits, on the arm that never stages.

EnsureRing is now the last term evaluated. The header splits the four cheap
terms out as StagingTermsExceptRing; ShouldStageH2D is spelled as
StagingTermsExceptRing(in) && in.ring_available and stays the single authority
the truth table gates. A 480-input case walks the split, and the SECOND review
round replaced what that case asserts; see below.

RED FIRST, on the board, with the selector nobody had run. At the test
commit c794b5dda,
VT_ROCM_MANAGED_ALLOC=1 test_backend_cross_device -tc='*pinned bounce*'
reported 1 case / 4 assertions, 1 failed, Status: FAILURE!, exit 1,
ring_bytes=268435456, pinned H2D case ran on a ROCm board: 1. At the fix
commit it reports 4 / 4 with ring_bytes=0. Binaries proven changed by md5 on
both arms.

The capture guard was in the wrong place. in.stream_capturing was
hardcoded false while a separate early return false did the work, so the
term the truth table gates was a value production never supplied. The probe now
feeds the field and the predicate IS the guard. StreamIsCapturing's own return
value is still not measured on a board, and the spec says why rather than
implying otherwise: a pageable asynchronous H2D is itself illegal inside a
capture region, so a case there would measure HIP's refusal and not this guard.

The public documents owed a projection. docs/FEATURES.md still read "NO
TOKEN HAS COME OUT OF A GPU FOR THIS MODEL and none is claimed". It now records
what ROCm produced and is explicit that the claim is LIVENESS, not correctness:
this architecture has no GPU oracle at all, so there is no token gate, no
denominator and no admissible speed comparison. The CUDA half of that sentence
is untouched and still owed elsewhere. docs/USAGE.md's checkpoint cell names
the ROCm arm on the same terms.

A discrepancy the tree already resolved. The spec said 71.96 GiB "exceeds
the 33.27 GB hipMemGetInfo total" and declined to resolve it.
.agents/environment.md resolved it already: this part reports 96.000 GiB of
VRAM since the 2026-09-11 firmware change, and 33,270,497,280 B is the box's
HOST RAM. rocm-host-residency-after-upload.md §6a read one for the other and
this spec inherited it; both are corrected.

The model still produces tokens after the repair, confirmed once rather than
re-measured: three runs, completion_tokens=32 finish_reason=length each, exit
0. The steady-state decode figure this row reports is a RANGE, 5.0-5.3 tok/s
over nine samples from six independent process launches (5.002 to 5.291, 5.8%
max-to-min); an earlier two-launch reading quoted 5.27-5.29 at 0.34% and that
precision does not reproduce.

Round 2: a drift guard that could not detect drift

A second fresh review returned FAIL on three claims, and none of them was a
defect in shipped behaviour: the release and staging logic is unchanged here.

The 480-input case was a TAUTOLOGY. It asserted
StagingTermsExceptRing(in) == ShouldStageH2D(in) with ring_available held
true. The decision is DEFINED as the helper AND that flag, so that assertion is
X == (X && true) -- true for any definition of the helper, deleted terms
included. Measured on strix:gpu0: deleting dst == kDevice, and separately
deleting bytes >= chunk_bytes, each left that case 1443/1443 SUCCESS. Both
mutants died to the PRE-EXISTING five-term truth table instead. The case now
computes its expectation in the test file from the four inputs, as a sequence of
refusals rather than as a conjunction, and checks BOTH expressions against it,
so a deleted term fails the case itself.

Re-measured at 92d3305cd in the same clone. Clean unit binary
32b4cade5a4bba7d2060f6086a681dde, 10 / 2007 SUCCESS, the case alone 1 / 1923
SUCCESS. Deleting in.dst == PtrKind::kDevice && gives binary
c7f00ede469d373502f961347e5b7361 and -tc=*cheap terms* now reports FAILURE
at test_rocm_pinned_h2d.cpp:363; deleting in.bytes >= in.chunk_bytes gives
c1e2d3d36df481a2cc49c7f091c1210b and the same case fails at the same line.
Both restored builds hash back to 32b4cade5a4bba7d2060f6086a681dde and run
10 / 2007 SUCCESS. The spec, the header comment and this body all carried the
false claim; all three are corrected. A drift guard that cannot detect drift is worse
than none, because it gets cited as protection.

docs/FEATURES.md contradicted itself inside one cell. The cell recorded
that a GPU has produced tokens on ROCm and five sentences later still said "WHAT
RUNS IS EXACTLY THIS: --device cpu ... and no more". The sentence now names
both, and keeps the ROCm arm as LIVENESS and never parity.

The board-size misread survived one "corrected everywhere". §6 Risks still
read "a 33.27 GB board". 33,270,497,280 B is this box's HOST RAM;
hipMemGetInfo reports 96.000 GiB since the 2026-09-11 firmware change
(.agents/environment.md:89-92). Fixed, and both specs were swept for further
instances: grep -rn '33\.27\|33,270,497,280' .agents docs now shows the
number only where it is labelled host RAM or inside a correction paragraph that
names the misread.

FOLLOWING_AGENTS_PROTOCOL

Following-Agents-Protocol: true
AI-Assisted: true
Assisted-by: AGENT:claude-opus-5-1m [claude-code]

… ring The 67.56 GiB Qwen3.8-Flash-Next UD-IQ1_S still produces no token on gfx1151 after the two host-residency fixes landed, and the compute thread is still uninterruptible in svm_range_set_attr for about three quarters of its wchan samples. Releasing the spent source pages was necessary and is not sufficient, so the remaining suspect is the SHAPE of the transfer: RocmBackend::Copy hands the ROCr runtime one multi-GiB pageable, file-backed, CIFS-backed range and the runtime has to make all of it resident to DMA out of it. This spec takes llama.cpp's ring at the recorded pin -- four pinned buffers of 64 MiB with an event each, llama-model-loader.cpp:1440, :1449, :1496-1516 and :1591-1642 -- and states the one place it differs from ours, which is that the oracle takes that ring only when it is NOT using mmap. What we port is the shape, not the call, and the spec says so rather than claiming a parity it does not have. The reach is enumerated before the code because this is a shared seam and this row has already shipped a seam change whose blast radius nobody wrote down. One override of Backend::Copy is touched, no other backend is, and the staged path needs five terms to hold at once, so a 4 KiB norm keeps the existing single call byte for byte. FOLLOWING_AGENTS_PROTOCOL Following-Agents-Protocol: true AI-Assisted: true Assisted-by: AGENT:claude-opus-5-1m [claude-code]

…r and predicate The decision behind the bounce ring goes in a HIP-free header and is table-tested on a runner with no AMD GPU, which is the same split include/vt/rocm/rocm_arch.h already makes for ResolveMemoryPolicy and for the same reason: a ring that silently never engages is indistinguishable from one that works. The bytes are identical either way, the model loads either way, and only the host residency differs, which no gate in this tree reads by accident. So the chunk arithmetic, the cross-call slot reuse and the five-term predicate are gated here, and the half that cannot be gated here -- that RocmBackend::Copy actually calls any of it -- gets a device case in test_backend_cross_device that asserts the instrument beside the bytes. That case is RED at this commit: nothing calls the ring yet, so staged_copies stays at zero. Eight mutations were run against the pure half and each convicted, with the binary proven changed by md5 on every arm: the chunk size widened to the whole buffer, the slot wait deleted, each of the four predicate terms deleted, the cross-call in-flight state reset, and the knob parse made to answer 0 on a typo instead of the default. FOLLOWING_AGENTS_PROTOCOL Following-Agents-Protocol: true AI-Assisted: true Assisted-by: AGENT:claude-opus-5-1m [claude-code]

…d pinned ring on ROCm RocmBackend::Copy was one hipMemcpyAsync handed a multi-GiB pageable, file-backed, CIFS-backed source. To DMA out of a pageable range the ROCr runtime has to make all of it resident and describe it to the KFD, and that is where gfx1151 sits: 153 of 196 and 148 of 198 wchan samples in svm_range_set_attr across two 1200 s runs that never produced a token, with the two host-residency fixes already landed and the source pages already released. A copy of 64 MiB or more, from unregistered host storage to device memory, on a stream that is not capturing, now goes through four pinned 64 MiB buffers with an event each: wait on the slot, fill it, upload from it, record, advance modulo four. Host residency for the transfer is 256 MiB for a model of any size, which is llama.cpp's own bound at the recorded pin. Everything else takes the previous single call byte for byte -- every copy under one chunk, every readback, every D2D, every already-pinned or managed source, and anything inside a graph capture. VT_ROCM_PINNED_H2D_MIB=0 restores the old path in the same binary, so the load-time A/B is one build rather than two. The pointer classifier reads TWO shapes of "the runtime does not know this pointer", an error return and a success with null host and device pointers, because HIP has answered it both ways across versions and reading only one of them would leave the ring silently disengaged with identical bytes and an unchanged residency -- which is the failure this row has already shipped twice in the shape of a selector that matched nothing. FOLLOWING_AGENTS_PROTOCOL Following-Agents-Protocol: true AI-Assisted: true Assisted-by: AGENT:claude-opus-5-1m [claude-code]

…n gfx1151, and the same binary without it does not The gate counts, the red, the mutations and the model gate are all measured now, so the spec stops saying RECORDED AT §7 and says what was read. The model gate is met. On strix:gpu0 the 67.56 GiB Qwen3.8-Flash-Next UD-IQ1_S produced three generations of 32 tokens, exit 0, no deadline kill, and the compute thread spent none of its 122 wchan samples in svm_range_set_attr. The same binary minutes later with VT_ROCM_PINNED_H2D_MIB=0 reproduced the recorded wedge to within noise -- 135 of 197 samples in svm_range_set_attr, device memory stuck at the same 29.69 GiB plateau, killed at the 1200 s deadline with no token. One binary, one boot, the knob read at runtime, so the ring is the cause rather than a coincidence of the day. Three things this deliberately does not claim. There is no throughput comparison, because no oracle ran this workload. There is no load-time verdict, because the two arms read the checkpoint at different page-cache temperatures. And the warm decode pair is TWO samples, not three, because --repeat 3 gives one cold generation that carries the whole 72 GiB device staging; the number is recorded as thin instead of being presented as a spread it is not. VT_ROCM_PINNED_H2D_MIB is a new config key, so ENVIRONMENT.md and USAGE.md carry it, including the fact that turning it off is what makes this artifact stop producing tokens. FOLLOWING_AGENTS_PROTOCOL Following-Agents-Protocol: true AI-Assisted: true Assisted-by: AGENT:claude-opus-5-1m [claude-code]

…decode number four samples The third arm ran the same binary and the same ring over a local copy of the same three shards, and it answers the question §6a of the residency spec left open. The first generation falls from 695.871 s to 38.438 s and the D-state histogram collapses from 115 page-cache waits to nothing, so 657 of those 696 seconds were reading the file over CIFS and the ring moves the 72 GiB checkpoint onto the board in about 38 s. The confound was never the cause of the wedge -- the off arm wedges on the same mount the on arm succeeds on -- but it was almost all of what was left. It also fixes the thin part of the previous record. The steady-state decode number is now four samples across two independent process launches on two different source filesystems, 5.291 / 5.273 / 5.274 / 5.275 tok/s, a 0.34% spread. Three repetitions inside one handle would have satisfied the row's rule more weakly than this does. FOLLOWING_AGENTS_PROTOCOL Following-Agents-Protocol: true AI-Assisted: true Assisted-by: AGENT:claude-opus-5-1m [claude-code]

… on the managed arm The cross-device pinned-bounce case already printed `ring_bytes` on both arms and asserted it on only one. On the managed arm -- gfx1103, or any board run with `VT_ROCM_MANAGED_ALLOC=1` -- the destination is already device addressable, every copy is direct, and the case was green while the instrument printed `ring_bytes=268435456`. A ring that is allocated and never used changes no byte, so nothing else in this file can see it. It is not free on the board that shows it. `.agents/environment.md:95-100` measures gfx1151's managed ceiling as bounded by HOST RAM rather than by the 96 GiB carve: 27 GiB reached against 29.3 GiB available. Pinned host memory comes out of exactly that bound, so 256 MiB of it is charged to the resource that decides whether the model fits, on the arm that never stages. This case fails until `RocmBackend::StagedCopy` stops allocating the ring before it knows the copy will use it. FOLLOWING_AGENTS_PROTOCOL Following-Agents-Protocol: true AI-Assisted: true Assisted-by: AGENT:claude-opus-5-1m [claude-code]

…t will use it `StagedCopy` evaluated `in.ring_available = EnsureRing(chunk)` before it asked `ShouldStageH2D`, so the first copy of 64 MiB or more on a non-capturing stream built the ring whatever the pointers turned out to be. Any ROCm process that performs one such copy permanently allocated 256 MiB of pinned host memory and four events even when the copy then took the direct path -- D2D, an already-pinned source, or a managed destination. Measured rather than reasoned: `VT_ROCM_MANAGED_ALLOC=1` on gfx1151 printed `staged=0 direct=3 chunks=0 max_chunk=0 ring_bytes=268435456` and the case still passed, because the managed arm asserted nothing about `ring_bytes`. The instrument printed the defect and nothing read it. That assertion now exists and is what this commit turns green. It is not a free 256 MiB on the boards that show it. `.agents/environment.md` measures gfx1151's managed ceiling as bounded by HOST RAM, 27 GiB against 29.3 GiB available, so pinned host memory is charged to the resource that decides whether a checkpoint fits, on the arm that never stages. `ShouldStageH2D` stays the single authority on the decision. It is now spelled as `StagingTermsExceptRing(in) && in.ring_available`, production asks the cheap half first and calls `EnsureRing` only when it passes, and a new truth-table case asserts the two agree over 480 inputs so the split cannot drift. The same restructure un-deadens `stream_capturing`. It was hardcoded false while a separate early return did the work, so the term the truth table gates was a value production never supplied. The probe now feeds the field and the predicate is the guard, which is what the spec always said it was. FOLLOWING_AGENTS_PROTOCOL Following-Agents-Protocol: true AI-Assisted: true Assisted-by: AGENT:claude-opus-5-1m [claude-code]

…ct three figures it got wrong `docs/FEATURES.md` still read "NO TOKEN HAS COME OUT OF A GPU FOR THIS MODEL and none is claimed". That is false for ROCm since 2026-09-13: `--device auto` on gfx1151 returns 32 tokens with `finish_reason=length` on three launches out of three, fluent and prompt-dependent. What is claimed is LIVENESS and nothing more. This architecture has no GPU oracle at all -- llama.cpp aborts in `build_delta_net_chunking` before reading a byte and no vLLM revision implements `qwen4_exp` -- so there is no token gate, no denominator, and no admissible speed comparison, and the sentence says so. The CUDA half of the old claim is untouched and still owed by QSADEV and #2423. `docs/USAGE.md`'s checkpoint cell enumerated `--device cpu` and `--device cuda` and now names the ROCm arm on the same terms. Three figures are corrected rather than carried: The decode number. This spec quoted "5.27-5.29 tok/s, spread 0.34%" off four samples across two process launches. A fresh review launched it three more times and read 5.002, 5.183 and 5.097. Nine samples across six launches run 5.002 to 5.291, a 5.8% spread, so the honest quotation is the range 5.0-5.3 tok/s with the launch count beside it. The board's memory total. `rocm-host-residency-after-upload.md` section 6a said 31.88 GB climbed toward "the board's 33.27 GB total" and this spec's section 7 inherited the number and declined to resolve the discrepancy it created. `.agents/environment.md` already resolved it: since the 2026-09-11 firmware change this part reports 96.000 GiB of VRAM, and 33,270,497,280 B is the box's HOST RAM. Both specs now say so, and 71.96 GiB of 96.000 GiB needs no explanation. The two guarantees the repair moved. Section 3b term 3 promised `VT_ROCM_MANAGED_ALLOC=1` was unchanged while the ring was still built on that arm, and section 6 placed the capture guard in a truth-table term production never fed. Both now describe the code, and section 6 states plainly what is still not measured on a board. Section 5a records the repair wave's red, its green, its mutation and the one confirmation run that the model still produces tokens. FOLLOWING_AGENTS_PROTOCOL Following-Agents-Protocol: true AI-Assisted: true Assisted-by: AGENT:claude-opus-5-1m [claude-code]

… Strix oracle suite `scripts/agent-preflight.sh` fails its "tools suites" gate on this dev box with 49 failures and 1 error from `tests/tools/test_strix_vllm_oracle.py`. None of them mention disk. The cause is `tools/bench/strix_vllm_oracle/worker.py:257` raising `ValueError("disk headroom exhausted")`, which the test helper turns into an `assertEqual(returncode, 0)` whose message carries the traceback and whose name does not. It reproduces on a PRISTINE `git archive` of `origin/main` `ee0644eab`, to the same 49 and 1, on a host at 97% full. So it is an environment condition an unrelated row's preflight reads as its own red, which is the shape this repository calls an instrument whose failure looks like a result. A headroom guard that cannot run should SKIP with its reason named, as preflight already does for its five argument-starved gates. This file rides in the ROCm chunked-H2D pull request because that is the flow that found it, and because a filed gap is worth more tracked than left untracked in a worktree. It changes no code and belongs to no other change on this branch; drop the commit if it is preferred as its own pull request. FOLLOWING_AGENTS_PROTOCOL Following-Agents-Protocol: true AI-Assisted: true Assisted-by: AGENT:claude-opus-5-1m [claude-code]

… term, and correct what it was said to prove The 480-input case that landed with the eager-`EnsureRing` repair asserted `StagingTermsExceptRing(in) == ShouldStageH2D(in)` with `ring_available` held true. The decision is DEFINED as the helper AND that flag, so that assertion is `X == (X && true)`: true for any definition of the helper, deleted terms included. A fresh review measured it on `strix:gpu0`: deleting `in.dst == PtrKind::kDevice` from the helper, and separately deleting `in.bytes >= in.chunk_bytes`, each left the case 1443/1443 SUCCESS. Both mutants died to the PRE-EXISTING five-term truth table instead, three and two assertions elsewhere in the binary, so a drift guard that cannot detect drift was being cited as protection. The case now computes its expectation in the test file from the four inputs, as a sequence of refusals rather than as a conjunction, and checks BOTH expressions against it. A term deleted from either one now fails this case itself. The spec, the header comment and the pull request body each claimed the old case proved that; all three are corrected, and the spec's attribution of a whole-binary failure count to the case just added is withdrawn by name. `docs/FEATURES.md` contradicted itself inside one cell: it recorded that a GPU has produced tokens for this model on ROCm and five sentences later still said "WHAT RUNS IS EXACTLY THIS: `--device cpu` ... and no more". The sentence now names both arms and keeps the ROCm one as liveness and never parity, because this architecture has no GPU oracle and there is no token gate. The spec's section 6 still read "a 33.27 GB board" after the commit that claimed to correct that misread everywhere. 33,270,497,280 B is this box's HOST RAM; `hipMemGetInfo` reports 96.000 GiB since the 2026-09-11 firmware change (`.agents/environment.md:89-92`). Fixed, and both specs were swept: the number now appears only where it is labelled host RAM or inside a correction paragraph that names the misread. No release, staging or predicate logic changes here. The guarantee itself was already gated, by the five-term truth table this case sits beside. FOLLOWING_AGENTS_PROTOCOL Following-Agents-Protocol: true AI-Assisted: true Assisted-by: AGENT:claude-opus-5-1m [claude-code]

mudler force-pushed the row/MODEL-MM-QWEN4-EXP-ROCM-CHUNKED-H2D branch from 7125d70 to 33a1eaa Compare September 13, 2026 07:30

mudler added 7 commits September 13, 2026 07:43

mudler force-pushed the row/MODEL-MM-QWEN4-EXP-ROCM-CHUNKED-H2D branch from a4582b6 to fdfbeb1 Compare September 13, 2026 07:43

mudler force-pushed the row/MODEL-MM-QWEN4-EXP-ROCM-CHUNKED-H2D branch from fdfbeb1 to 933039a Compare September 13, 2026 07:46

mudler added 2 commits September 13, 2026 08:00

mudler force-pushed the row/MODEL-MM-QWEN4-EXP-ROCM-CHUNKED-H2D branch from 8e1f0ac to 92d3305 Compare September 13, 2026 08:17

mudler merged commit 92d3305 into main Sep 13, 2026
1 check failed

mudler deleted the row/MODEL-MM-QWEN4-EXP-ROCM-CHUNKED-H2D branch September 13, 2026 08:27

Navigation Menu

feat(MODEL-MM-QWEN4-EXP): stage a large pageable H2D through a bounded pinned ring on ROCm - #3180

feat(MODEL-MM-QWEN4-EXP): stage a large pageable H2D through a bounded pinned ring on ROCm#3180
mudler merged 10 commits into
mainfrom
row/MODEL-MM-QWEN4-EXP-ROCM-CHUNKED-H2D

localai-org-maint-bot commented Sep 13, 2026 •

edited

Loading

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

Conversation

localai-org-maint-bot commented Sep 13, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

The reach, enumerated before the code was written

Red, green, and the mutations

The model gate, which is the deliverable

The repair wave: what a fresh review found, and what it cost

Round 2: a drift guard that could not detect drift

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

localai-org-maint-bot commented Sep 13, 2026 •

edited

Loading