context : preserve quantized block sizes in device state I/O

[arc-uri-el]

Overview

Device checkpoint I/O converts byte counts into 1D tensor dimensions with bytes / ggml_element_size(tensor). ggml_element_size returns ggml_type_size, which is the byte size of a whole quantization block. For Q8_0 and Q4_0, the resulting view has one thirty-second of the intended logical elements.

Multiply by ggml_blck_size in the writer, reader and fragmented-copy subview. No public API, serialization metadata, allocation policy, kernels or sampling code changes. The existing save/load test gains a fragmented-to-compact round trip that compares the resulting host state byte-for-byte.

The current-base Q8_0 and Q4_0 tests segfault during the on-device scatter restore on both CPU and Radeon Vulkan. The isolated patch makes both pass. F32 passes before and after. This was investigated on a Strix Halo machine while qualifying quantized prompt checkpoints, not inferred from an unrelated device.

Measurements

Device:     Ryzen AI MAX+ 395 / ASUS ROG Flow Z13 GZ302EAC
Memory:     128 GB LPDDR5X
Power:      AC; performance platform profile and CPU governor; GPU clocks auto
Firmware:   BIOS 304; PPT settings 80/92/93 W; VRAM 512 MiB
Kernel:     Linux 7.3.0-rc1; ttm.pages_limit=30408704; GTT 116 GiB
Backend:    CPU and Radeon 8060S Vulkan RADV, Mesa 26.2.0
Build:      GCC 15.2, Release, shared libraries, native CPU, Vulkan, OpenMP; -j8
Baseline:   7449a0fe9710ab584c5f9a6d25e7a31eea2708b8
Change:     fc349bd785d8bd878235039d8e35a186d872d0cb
Model:      test-llama-archs generated llama-dense; separate official Qwen3.8-27B Q4_K_M control

The baseline and candidate were built separately in this session with the same settings and no compiler warnings. The contribution's attribution was subsequently shortened; its complete source tree is byte-identical to the tested tree. Runtime libraries were selected from their corresponding build directories. The later master merge 99a40a3e6 changes speculative replay, not these I/O or test lines.

Baseline / after:

Backend	KV	Baseline	Candidate
CPU	F32	pass	pass, all 9 tests
CPU	Q8_0	SIGSEGV at device scatter restore	pass, all 9 tests
CPU	Q4_0	SIGSEGV at device scatter restore	pass, all 9 tests
Vulkan0	F32	pass	pass, all 9 tests
Vulkan0	Q8_0	SIGSEGV at device scatter restore	pass, all 9 tests
Vulkan0	Q4_0	SIGSEGV at device scatter restore	pass, all 9 tests

The existing host save/load and generated-token checks pass before the failing baseline scatter case. The candidate passes those checks, the host/device scatter byte comparison and the added compact-destination comparison. A second, independently generated synthetic llama fixture also reproduces the quantized failure and passes with the fix.

Reproduce with the existing tools (run in a disposable working directory because the save/load test writes dump_state.bin):

cmake --build build --target test-llama-archs test-save-load-state
mkdir -p fixtures
build/bin/test-llama-archs -a llama -s 1234 -o fixtures
build/bin/test-save-load-state -m fixtures/llama-dense.gguf \
  -c 256 -b 64 -ub 64 -n 8 -t 1 -ngl all -dev Vulkan0 \
  -ctk q8_0 -ctv q8_0 -fa on

Repeat with q4_0. CPU control: -ngl 0 -dev none. F32 control: -ctk f32 -ctv f32 -fa off.

Correctness:

On the unchanged official ggml-org/Qwen3.8-27B-GGUF Q4_K_M, separate baseline/candidate Vulkan runs also produced byte-identical prompt IDs, nine complete logit rows (prefill plus eight greedy decode steps) and selected IDs for each of this repository's prose, code, structured and numeric corpora. Each corpus was repeated and truncated to 512 input tokens; both used Q8_0 KV, FA on and the same model file. This is a bounded fresh-inference control, not a claim that every state/backend path is covered.

No throughput improvement is claimed. llama-bench, perplexity and the full backend-op suite were not run for this I/O-only patch; the acceptance evidence is the failing and passing state round trips and exact output controls above.

Requirements

• I have read and agree with the contributing guidelines.
• This change is justified by the CPU and actual Strix Halo Vulkan state measurements above. No GPU-specific behavior is inferred for untested backends.
• AI usage disclosure: AGENT-AUTHORED. GPT 6 Astra investigated the failure, isolated the patch, extended and executed tests, and prepared this PR at the explicit direction of the submitting account's owner. The submitting account owns review and follow-up. Original broader local investigations used LLM assistance as well. No human Reviewed-by or Tested-by attestation is claimed.
• What was NOT verified: other GPU backends, other operating systems, all model architectures, full occupied long contexts, perplexity and the full backend-op/performance matrix. The test proves the stated sizing and restoration boundary; it does not claim global correctness.