feat(QUANT-GGUF-IQ4_NL): IQ4_NL keep-quant GEMM on ROCm

[localai-org-maint-bot]

IQ4_NL reached the GGUF reader, the CPU dequantizer and the CPU keep-quant dot,
and no accelerator multiplied it. On ROCm the refusal was terminal, because the
backend is discrete and an unsupported dtype cannot fall back to a host kernel
that would follow device pointers.

This blocks a shipped artifact rather than a hypothetical one. Every published
unsloth/Qwen3.8-Flash-Next-GGUF quant stores 91,465,564,160 elements in IQ4_NL,
read from the three shard headers over HTTP range requests: the 20M-entry
n-gram table and the 48 ffn_down_exps.

What lands

The ROCm IQ4_NL keep-quant GEMM, as two kernels rather than one. An expert
tower reaches the grouped provider and not the single-matrix one, so
GroupedIQ4NLK is the arm the shipped checkpoints actually need and IQ4NLGemmK
covers the non-grouped path beside it. Admitting the encoding with only the
latter would throw at the first expert forward with the model already resident,
which is the failure DeviceKeepQuantSupported exists to prevent, so the
predicate and both kernels move together.

DotIQ4_NL is a 1:1 port of cpu_quant_dot.cpp VecDotIQ4_NLQ8_0, itself llama.cpp
b10451 quants.c:1254. It keeps upstream's association, d * (sumi1 + sumi2),
where the scale product is formed before the integer sum is folded in. That is
the opposite order from DotQ8_0 beside it, and it is what makes the GEMM
bit-reproducible against the oracle.

The activation encoding is Q8_0, not the Q8_K every neighbouring i-quant uses.
ROCm already had that path, so this is a new dot in existing machinery rather
than new machinery. CUDA genuinely lacks it and is unchanged and still owed.

The gate, red and green

Run on the hardware it is for, under an rc lease on strix:gpu0 (gfx1151, ROCm
7.2.4, HIP 7.2.53211), built in the lease from a --depth 1 clone with
git rev-parse HEAD asserted equal to the commit under test and
git status --porcelain at 0 bytes.

Run	Head	Result
RED	3c529c8	47 cases / 45 passed / 2 failed, 84040 assertions / 0 failed
GREEN	6f77c70	47 / 47 / 0 failed / 0 skipped, 84044 assertions / 0 failed

The red is the evidence and not a mishap. Both failing cases were the two new
IQ4_NL ones, and both threw by name rather than computing a wrong answer: zero
failed assertions against two failed cases is the signature of a route gap. The
cause was the wrapper in rocm_quant_dot.hip, whose delegation allowlist and
whose IsRocmKeepQuantSupported both excluded IQ4_NL, leaving the kernels in the
GDN provider unreachable. The third commit fixes exactly that.

The assertion count rising by four across the pair is what shows the format
executed rather than being skipped; in the red run those cases threw before
their checks ran. 0 skipped is asserted on the green run, not assumed. The
build was clean at -Werror with 0 warnings.

What this does NOT claim

No throughput, latency or memory number is claimed. AGENTS.md admits no
performance result from an arm whose token gate has not passed, and the gfx1151
token gate currently reads FAIL at 3 of 6 on a simpler model.

IQ4_NL alone does not make Qwen3.8-Flash-Next run on ROCm. NINE qwen4_exp
operations still have no ROCm arm: kQwen4ExpPleConv, kQwen4ExpPleGate,
kQwen4ExpGatedResidual, kQwen4ExpGatedResidualWriteBack, kQwen4ExpQsaCompress,
kQwen4ExpQsaGatherAttention, kRmsNormGroup, kIndexSelect and kIndexCopy. They
HARD-REFUSE on this board rather than falling back: ReferenceTierEligible gates
on DeviceMemoryIsHostAddressable(), which is unified_memory_, which
ResolveMemoryPolicy computes as managed_alloc || (pageable_memory_access &&
integrated) — and gfx1151 reports pageableMemoryAccess 0, so since the #2511
narrowing that is false. The repair that stopped the gfx1151 GPU hang also
removed the CPU reference tier on this board.

Review history, and what is still owed

A fresh review returned FAIL on seven findings. The kernel substance passed that
review on its own terms -- the arithmetic against llama.cpp b10451
quants.c:1254, every stride at sizeof(BlockIQ4_NL), route and refusal agreeing
on both arms, and four mutations behaving -- and every finding was at the edges
of the change. All seven are repaired by a fresh implementer:

• The blocking one was a red gate the first wave never ran. Widening
  DeviceKeepQuantSupported for kROCM reds tests/vllm/test_gguf_keep_quant.cpp,
  which restates the ROCm admitted set independently and as a total. The row's
  declared gate was the ROCm kernel suite, which cannot see a loader predicate.
  The predicate is NOT narrowed; the test's expectation moves, gemm_kept 22 to
  24.
• The spec called the association order d*(s1+s2) load-bearing and pinned it
  with nothing: reassociating survived a fully green gate. Every gate this row
  had was an NMSE band at 5e-4, and the reassociation moves the result by about
  1e-7 relative. It is now pinned bit for bit by a case that runs on every
  device registering kMatmulBTQuant, and the mutation is proven to red it at one
  ulp, 3240805058 against 3240805059, with the binary proved changed by md5.
• The fused MoE gate+up+SwiGLU arm serves IQ4_NL and was ungated; its fmts table
  is extended rather than the stray line deleted.
• The records, docs/FEATURES.md and one stale comment now describe the tree as
  it is, and the issue carries dated resolution evidence.

A FRESH SCOPED RE-REVIEW OF THOSE REPAIRS HAS NOT RUN. It was dispatched and
died immediately on a session rate limit, so nothing has independently read the
repaired head. The gate numbers the implementer reports were also taken at
0593ea4, BEFORE this branch was rebased onto origin/main 97cb696, so they
describe a different base than the one here. Both are owed before this merges,
and neither is a formality: the repairs touch a checker baseline and a test that
exists to pin a one-ulp guarantee.

A correction this pull request carries against itself

Two of the arms this row opened with were never its work, and the record says so
rather than quietly shrinking. The ROCm gather landed as #3097 while the spec
was being written. The CUDA dot was already landed as #2419, and that one was a
mistake rather than a race: the spec consulted IsCudaKeepQuantSupported, which
governs the 256-element Q8_K family, for a 32-element Q8_0-activation encoding
that cannot appear in it by construction. CUDA keeps a second predicate,
IsCuda32BlockKeepQuantSupported, for exactly that class.

Records

The QUANT-GGUF-IQ4_NL row understated the tree and now says what it does: the
reader arm, the dequantizer and the CPU dot all landed under #1989 while the
cell read INVENTORIED with every stage unset.

strix:gpu0 joins the fleet table, which predated the box, with its memory
measured under two rc leases. hipMallocManaged there is bounded by HOST memory
rather than by the firmware carve: a bounded probe that stops at its first
failure reached 76 GiB with plain hipMalloc and 27 GiB with hipMallocManaged,
against 29.3 GiB host-available. That resolves an ambiguity #2518 could not,
because its 58.000 GiB ceiling sat below both the 64.00 GiB carve and the 62 GiB
host RAM of the time and so never said which bound it hit. Raising the carve to
96 GiB lowered the managed ceiling, because host RAM fell with it.

The spec records two corrections against itself rather than applying them
silently. It first planned a ROCm gather arm and first recorded #3029 as a hard
dependency; #3097 landed both the gather and the ROCm codebook header while this
was being drafted, so the arm is struck and the dependency is downgraded to a
conflict surface.

FOLLOWING_AGENTS_PROTOCOL

Following-Agents-Protocol: true
AI-Assisted: true
Assisted-by: AGENT:claude-opus-5-1m [claude-code]