Speculative prefill - #27692

Open

kilofox wants to merge 22 commits into

ggml-org:masterfrom

kilofox:spec-prefill

Commits on Aug 25, 2026

scripts : add speculative prefill stress and evaluation scripts
kilofox
committed
96be5de
speculative-prefill : add dedicated draft device parameter
kilofox
committed
b3173fc
speculative-prefill : wire sparse prefill into server and CLI
kilofox
committed
4db32fa
speculative-prefill : fix attention capture, RoPE positions, and draft reuse
kilofox
committed
dd7c7e2
server : fix prompt throughput calculation for speculative prefill
kilofox
committed
57b8386
server : allow speculative prefill on text prompts when mmproj is loaded
kilofox
committed
c4f085a
server : use contiguous positions for speculative prefill tokens
kilofox
committed
e3c4aac
server : scale intermediate prompt progress throughput for speculative prefill
kilofox
committed
027adea
speculative-prefill : align server RoPE positions and set greedy lookahead in CLI
kilofox
committed
2973ac3

Commits on Aug 26, 2026

Commits on Aug 27, 2026

Merge remote-tracking branch 'upstream/master' into spec-prefill
kilofox
committed
29b0fd3

Commits on Aug 28, 2026

Commits on Aug 29, 2026

Commits on Sep 1, 2026

speculative-prefill : clamp default draft context to training limit
kilofox
committed
79acc58