You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
While updating three rows for #440 I found the README benchmark table has drifted beyond just those rows — both numbers and prose descriptions are stale in places. A full warm re-bench would refresh the whole table.
Qwen3-Coder 30B-A3B, CUDA -g -1: the row said "routed experts stream through CudaExpertSlotManager SLRU" — but the default path now auto-selects CPU-MoE (expert-cache capacity too small), a different code path. The number (29.4) was the per-token CPU-MoE rate, not SLRU. Both fixed in perf(cpu): batched CPU-FFN/MoE prefill — hoist Q6_K input re-quant, default flags on (#416) #440, but the description had been wrong for a while (predates this session — the auto-select behavior changed without a README update).
The other rows (full-GPU, op-offload MoE, CPU-only, Vulkan) weren't affected by #440 and were left as-is, but they date from earlier sessions and predate later perf work — so some have likely drifted too (numbers and/or notes).
on the reference machine (Zen 4 12c/24t + RTX 4070 Ti), then update the table numbers and audit the per-row notes for stale claims (like the SLRU one). Same-thermal, interleaved, per the #410 protocol.
Scope note: this is the sharpi-side number refresh; the llama.cpp gap tracking lives in #405. Related: #440 (the fix that surfaced this), #405.
While updating three rows for #440 I found the README benchmark table has drifted beyond just those rows — both numbers and prose descriptions are stale in places. A full warm re-bench would refresh the whole table.
Evidence of drift (found during #440)
-g -1: the row said "routed experts stream throughCudaExpertSlotManagerSLRU" — but the default path now auto-selects CPU-MoE (expert-cache capacity too small), a different code path. The number (29.4) was the per-token CPU-MoE rate, not SLRU. Both fixed in perf(cpu): batched CPU-FFN/MoE prefill — hoist Q6_K input re-quant, default flags on (#416) #440, but the description had been wrong for a while (predates this session — the auto-select behavior changed without a README update).The other rows (full-GPU, op-offload MoE, CPU-only, Vulkan) weren't affected by #440 and were left as-is, but they date from earlier sessions and predate later perf work — so some have likely drifted too (numbers and/or notes).
Proposed
Run the full warm sweep and reconcile every row:
on the reference machine (Zen 4 12c/24t + RTX 4070 Ti), then update the table numbers and audit the per-row notes for stale claims (like the SLRU one). Same-thermal, interleaved, per the #410 protocol.
Scope note: this is the sharpi-side number refresh; the llama.cpp gap tracking lives in #405. Related: #440 (the fix that surfaced this), #405.
🤖 Generated with Claude Code