Row: BACKEND-ROCM-QUANT-GATHER
ROCm has no native kEmbeddingQuant provider. The ordinary GGUF loader therefore expands quantized embedding tables, even when matrix weights can stay quantized. This consumes memory and leaves a backend capability gap on gfx1100.
Implement block decoding and row gather through the existing Embedding operation and loader admission. Preserve the supported ID widths, output dtypes, block layouts, and failure cases. Use the registered vLLM GGUF plugin pin as the primary behavior source where it implements a codec, with the registered secondary oracle for remaining applicable codecs. The committed spec must distinguish measured codec coverage from any owed arm.
Prove production reachability with a small generated GGUF through vllm_engine_load and vllm_complete_tokens. Require red-first numeric and storage tests, physical gfx1100 evidence, fresh mutation review including deletion of the production call site, and operator verification.
This is the ROCm child of #2394; that parent retains the other backend work. It does not depend on the quantized GEMM provider added by PR #2782. No CI changes are included.
Spec: .agents/specs/rocm-quant-gather.md, committed before implementation at 670e6d78ddf55231394748e0032939fd53dc56a5.
The operator approved a scoped Q8_K materializer amendment on 9 September 2026 UTC. The native public test reaches Q8_K and its load returns status 2. The new loader-focused test identifies the earlier reader refusal: GGML type 15 has no FindGgmlTraits entry. A separate downstream source hypothesis is that OwnGgufQuantBlocks requires a dot kernel after the gather policy admits the decoder-only format. Add an explicit embedding role at that call, preserve the default matrix dot requirement, and test both loader reachability and matrix refusal. The row spec commits this amendment before either shared materializer file changes. No shared dtype set, quantized dot provider, or PR #2782 work changes.
The operator approved the reader amendment before implementation: add only the existing Q8_K type-15 geometry, 256 elements and 292 bytes, preserving all other traits and validation. Test exact packed bytes through the reader, rerun the loader red to identify the downstream materializer refusal, and mutate both the type entry and geometry. The scope remains this row’s declared 19-codec public path.
Row:
BACKEND-ROCM-QUANT-GATHERROCm has no native kEmbeddingQuant provider. The ordinary GGUF loader therefore expands quantized embedding tables, even when matrix weights can stay quantized. This consumes memory and leaves a backend capability gap on gfx1100.
Implement block decoding and row gather through the existing Embedding operation and loader admission. Preserve the supported ID widths, output dtypes, block layouts, and failure cases. Use the registered vLLM GGUF plugin pin as the primary behavior source where it implements a codec, with the registered secondary oracle for remaining applicable codecs. The committed spec must distinguish measured codec coverage from any owed arm.
Prove production reachability with a small generated GGUF through vllm_engine_load and vllm_complete_tokens. Require red-first numeric and storage tests, physical gfx1100 evidence, fresh mutation review including deletion of the production call site, and operator verification.
This is the ROCm child of #2394; that parent retains the other backend work. It does not depend on the quantized GEMM provider added by PR #2782. No CI changes are included.
Spec:
.agents/specs/rocm-quant-gather.md, committed before implementation at670e6d78ddf55231394748e0032939fd53dc56a5.The operator approved a scoped Q8_K materializer amendment on 9 September 2026 UTC. The native public test reaches Q8_K and its load returns status 2. The new loader-focused test identifies the earlier reader refusal: GGML type 15 has no
FindGgmlTraitsentry. A separate downstream source hypothesis is thatOwnGgufQuantBlocksrequires a dot kernel after the gather policy admits the decoder-only format. Add an explicit embedding role at that call, preserve the default matrix dot requirement, and test both loader reachability and matrix refusal. The row spec commits this amendment before either shared materializer file changes. No shared dtype set, quantized dot provider, or PR #2782 work changes.The operator approved the reader amendment before implementation: add only the existing Q8_K type-15 geometry, 256 elements and 292 bytes, preserving all other traits and validation. Test exact packed bytes through the reader, rerun the loader red to identify the downstream materializer refusal, and mutate both the type entry and geometry. The scope remains this row’s declared 19-codec public path.