ggml-cpu: add opt-in Q2_0 VNNI64 four-row decode

[chris-lee-mc]

Summary

This draft reduces the Prism CPU proposal to the minimal Q2_0 backend change and its evidence:

• opt-in Phase 9 four-row AVX-512 VNNI decode in ggml-cpu;
• native Q2_0 storage retained as the fallback;
• minimal Q2_0 packer contract and implementation;
• deterministic logits/token parity comparator;
• concise real-model evidence.

Scope

Kept:

• ggml/src/ggml-cpu/CMakeLists.txt
• ggml/src/ggml-cpu/repack.cpp
• third_party/tbkern/include/tbkern/format.h
• third_party/tbkern/include/tbkern/q2_pack.h
• third_party/tbkern/src/q2_pack.c
• scripts/tbkern_compare.py
• docs/tbkern/phase9-four-row-vnni64.md
• docs/tbkern/phase9-prism-evidence.md

Removed from the proposal:

• Phase 1-8 narrative documents
• unused dispatch and standalone GEMV sources
• unrelated public TBKERN GEMV, GEMM, quantization, and arena APIs

Real-weight evidence

Model: Ternary-Bonsai-27B-Q2_0.gguf
Host: Google Cloud C3 highmem-8, Intel Xeon Platinum 8481C, 4 physical cores plus SMT, 8 ggml threads
Command: llama-bench -m MODEL -p 0 -n 16 -t 8 -ngl 0 -b 32 -ub 32

Baseline is unmodified Prism on the same model, host, command, and thread count:

Route	Run 1	Run 2	Run 3
Native Prism	1.09 tok/s	1.09 tok/s	1.09 tok/s
Phase 9 four-row VNNI64	1.52 tok/s	1.53 tok/s	1.53 tok/s

The historical matched set shows approximately a 40% target-decode improvement.

Correctness was bit-identical on the real model:

• logits SHA-256: 65d581397ae42288fe1115e7fa434589700cacd98edc67a750c0fba1765062f2
• generated-token SHA-256: 7a7748eacf971049271242b9d921628019d6c44698574e9301da9b8c88026381
• NMSE: 0
• top-1 mismatches: 0
• one-chunk PPL: 1.0645 +/- 0.03006

The selector remains opt-in because the recorded perplexity workload was slower even though PPL matched. Attention, norms, RoPE, softmax, GDN recurrence/convolution, KV cache, sampling, scheduling, and GPU execution remain Prism-native.

Validation

• based on Prism prism at 7529fda;
• git diff --check passes;
• scripts/tbkern_compare.py parses successfully;
• current Windows environment lacks CMake, GCC/Clang, and llama-bench, so a fresh rebuild of this cleanup commit is still required before marking this draft ready for review.

Raw model files and logs are not included.