fix(MODEL-MM-QWEN4-EXP): release a staged borrow's source pages, and stop prefaulting for a staging device

[localai-org-maint-bot]

fix(MODEL-MM-QWEN4-EXP): release a staged borrow's source pages, and stop prefaulting for a staging device

On gfx1151 the 67.56 GiB Qwen3.8-Flash-Next UD-IQ1_S loads with zero op
refusals and then wedges forever in svm_range_set_attr on a ~31 GiB host.
We call no SVM API ourselves, so the stall is host residency, and two sites
compound to make it.

Every borrowed GGUF span is prefaulted synchronously at load, and nothing
releases those pages once the device copy exists. For a keep-quant load the
source pages are the whole model: 65.488 GiB faulted in and read exactly once.
ReleaseHost()'s borrowed branch declines to touch them, arguing that clean
file-backed pages are reclaimable without our help. That holds against the page
reclaimer and fails against the KFD, which walks what this process has resident.

At the pinned llama-cpp oracle 10bf611e5 (b10451) the same model is
O(one tensor) resident during load and zero after: llama-model-loader.cpp
grows mmap_used only for tensors landing in a HOST buffer, then
unmap_fragment(0, mmap_used.first) — a real munmap — drops the whole mapping
for a fully offloaded model. Its prefetch is MAP_POPULATE plus an advisory
posix_madvise, with no synchronous touch loop.

Two changes.

MaybeReleaseStagedBorrowSource drops the spent source pages after the staging
upload, on a device whose kernels cannot read host memory, for a borrow whose
mmap_fd says the pages are file-backed and therefore re-faultable. The borrow
itself is untouched and stays a valid PROT_READ MAP_PRIVATE view, so a later
read re-faults identical bytes. It is called from behind the d_dev memo, which
is #1299's lesson: ResidentWeight runs about 1,361 times per forward step on
this checkpoint, and an unmemoized release would madvise away the pages the GPU
is about to read on every one of them. It synchronizes the queue first, because
the staging copy is an async stream copy.

THERE ARE TWO ResidentWeights AND THE CALL IS IN BOTH. The first revision of
this branch wired only the one in the unnamed namespace of qwen3_5.cpp, which
shadows the header one inside that translation unit. That covers the Qwen3.5
dense weights and, through KqResidentSlice and KqGrouped, the shared MoE
seam's keep-quant expert towers, which qwen4_exp does reach. It covers nothing
else in qwen4_exp: every attention, norm, hyper-connection, PLE and lm_head
weight stages through dense_attn::ResidentWeight instead, at 12 call sites in
qwen4_exp_forward.cpp, 10 in qwen4_exp_qsa_block.cpp, 7 in
qwen4_exp_ple_block.cpp and 1 in qwen4_exp_registry.cpp -- 30 in all, which
an earlier revision of the spec miscounted as 31 -- with no reference to
the qwen3_5.cpp function anywhere in that model. A fresh review measured the
consequence: the load peak fell from 27.67 GB to 8.18 GB and host RssFile still
climbed to 21.08 GB during the forward and stayed there. The same call now sits
in the same place in dense_attn_block.h — inside the d_dev memo, immediately
before AdoptDeviceBytesAsHost, with the platform's own
host_memory_is_device_addressable() answer passed in rather than re-derived.

THE SEAM IS SHARED, AND FOUR OTHER MODEL FAMILIES EXECUTE THE RELEASE. Counting
qwen4_exp's 30 call sites states the reach WITHIN this row's model, not the reach
of the change: dense_attn::ResidentWeight lives in a production header, and of
the 42 files that call it only four are qwen4_exp's. Nothing in an earlier
revision of this body, the spec or the commits said so. Spec §4a now carries the enumeration,
built by listing every caller of the two GGUF keep-quant borrow producers —
OwnGgufQuantBlocks and OwnGgufF16, the only writers of mmap_fd — and
tracing each forward to its consumption site. The release fires wherever a
BORROWED span with mmap_fd >= 0 meets a platform and a backend that cannot
dereference host memory, which is discrete CUDA and non-unified ROCm: dgx,
thor, orin and strix, the whole measurement fleet. Besides qwen4_exp that
is GLM-MoE-DSA (glm_moe_dsa_forward.cpp:162-186, :269, :334 — its MLA,
down_proj, router, norms, rope cache, embedding and head weights across 24
sites; its 228 routed-expert towers are refused one frame earlier by
GlmResidentExpertSlice), GLM5-Next (glm5_next_moe.cpp:243-245),
Muse-Glimmer (twelve sites in muse_glimmer.cpp plus muse_glimmer_mm.cpp:264)
and the Qwen3.5 DFlash draft head, whose EmbedTable() is the TARGET's kept-F16
GGUF table once model_loader.cpp rebinds it, at qwen3_dflash.cpp:597, :906,
:1851 and :1956. DeepSeek-V4 and Laguna have GGUF loaders and were checked
and EXCLUDED with the reason recorded, so nobody re-derives them.

NO TEST COVERS ANY FAMILY BUT QWEN4-EXP, AND THAT IS RECORDED AS OWED RATHER
THAN CLOSED. ISSUE-LOCAL-01M2CKN5516AKE7W2JVDV86Z8X owns it. The focused
harness's fake backend implements memory operations only and registers no
Embedding, MatmulBT or RmsNorm for kXPU, so a GLM5-Next or Muse-Glimmer
entry point refuses on a missing op long before residency is asked about, and
its process-global registrar cannot hold a second fake backend beside the
existing one. A case that called the seam directly with another family's tensor
shape would add a family's NAME and not a family's CALL SITE. Spec §4a also says
why the one repair that looked cheap was NOT taken: giving the header
ResidentWeight the VT_CHECK(!w.expert_streamed) its translation-unit-local
twin at qwen3_5.cpp:1181 already has would make the shared seam REFUSE a
weight it accepts today, and GLM5-Next stages three expert banks through it with
no test on any staging device — so the one-liner could remove a working path
instead of closing a hazard, and no gate here would say which. That hazard and
two more second-reader hazards are owed on
ISSUE-LOCAL-01M2CKNHVKJXT29XN8WM11KF6W.

SPEC §6 NOW NAMES A BINARY FOR EVERY SELECTOR AND PRINTS ITS COUNTS. A second
fresh review found that the focused gate line still carried a selector that
matches nothing — -tc=*DSA* against test_resident_weight_host_addressable
reports 0 cases, 0 assertions and Status: SUCCESS!, because DSA matches two
cases in test_backend_cross_device and the section named no binary. That is
the same failure -tc=*resident* already caused on this branch, twice in two
rounds, so every remaining gate line in the spec was swept for a third and the
sweep is reported rather than asserted: the four checkers are the only other
gate lines, none of them accepts or defaults to a selector, and each was run
with no arguments to confirm it refuses rather than passing empty. Measured on
strix:gpu0 under rc job f8a6ceec-d8bf-4e99-b6e4-3b2cbf71f8b9: the focused
binary is 28 cases / 139 assertions whole, -tc=*release* 9/52, -tc=*prefault*
2/11, -tc=*dense_attn* 3/19, and test_backend_cross_device is 60/84833 whole
with -tc=*DSA* 2/273.

GgufPrefaultForDevice gives the prefault the same device term
QuantRepackForDevice got in #2406, and GgufLoadPolicy::FromEnv carries it
into the field the loader reads; before this, every call site passed a literal
true. VT_GGUF_PREFAULT and vllm_cpp.mmap.prefault are answered before the
device is consulted, so the same-binary A/B stays reachable on the device the
default narrows.

The stale reason in RocmPlatform::residency_policy() is corrected in the same
change. It still said that freeing the host copy on a unified part would free the
only copy; #2511 falsified that, and leaving it in place is how this defect
survived. The flag itself stays false — but NOT because flipping it would be
inert, which is what an earlier revision of this body and that comment both
claimed. release_host_weights_after_upload has three readers, not two.
ShouldReleaseHostWeights and ShouldInterleaveLoadStream
(platforms/interface.h:88-96) do also require marlin_committed, which no ROCm
path sets. DirectDeviceLoadEligible (qwen3_5_dense_weights.cpp:146-180) does
not: it returns the flag as its last term and gates StageAndReleaseLoadedDense
at :1205-1212 for every Qwen3.5-dense safetensors load, behind
needs_weight_staging(), which is true on ROCm. Flipping the bit WOULD change
behaviour on this board, on a path nobody has measured there. That is the reason
it stays false, and the enumeration is the whole of
grep -rn release_host_weights_after_upload src include tests.

Two preconditions of the helper are not convicted by this tree's harness and are
recorded rather than chased. Deleting the backend's
DeviceMemoryIsHostAddressable() term leaves the focused suite green, because
the fake backend answers false unconditionally and no case presents the
disagreeing pair. Deleting backend.Synchronize(queue) leaves it green, because
HostBackend::Copy is a synchronous memcpy that cannot express a live DMA.
Both are owed on ISSUE-LOCAL-01M2CCNA0S74WT5WBV50B3VD0W; neither test was
weakened or contorted to manufacture a conviction.

THE MODEL GATE IS MEASURED AND THE STALL SURVIVES BOTH FIXES. On strix:gpu0
(gfx1151, ROCm 7.2.4, 30 GiB host) the 67.56 GiB UD-IQ1_S produces no token with
the release wired into both arms. Two runs, each killed at a 1200 s deadline:
peak VmHWM 25.99 and 26.05 GiB, peak RssFile 20.22 and 20.35 GiB, peak
RssAnon 5.55 GiB both, peak device memory 29.69 GiB of the board's 30.99 GiB
both. Host RssFile during the forward is essentially unchanged from the 21.08
GB the one-arm build showed. Sampling every thread every 6 s, the uninterruptible
thread is in svm_range_set_attr for 153 of 196 samples in run 1 and 148 of 198
in run 2, in folio_wait_bit_common for 41 and 47. Device memory was read from
/sys/class/drm/card0/device/mem_info_vram_used because rocm-smi is not on
PATH in the leased container; the earlier "717 MB of 103 GB" figure does not
describe this board and is withdrawn. The artifact was asserted HIP-linked before
it was timed. No throughput or latency number is recorded, because no token was
produced.

Chunked H2D through a pinned bounce buffer (llama.cpp's 4 x 64 MiB shape) is
deliberately NOT built here. That result makes it REQUIRED rather than optional,
and it has its own issue: it touches every staged weight on every backend and
needs its own measurement.

FOLLOWING_AGENTS_PROTOCOL

Following-Agents-Protocol: true
AI-Assisted: true
Assisted-by: AGENT:claude-opus-5-1m [claude-code]