server: on-disk KV cache (--kv-cache-dir) — restore a conversation instead of re-prefilling it - #16

server: on-disk KV cache (--kv-cache-dir) — restore a conversation instead of re-prefilling it#16

dzannotti wants to merge 4 commits into

masterhalo-box/llama.cpp:masterfrom

kv-disk-cache-haloboxhalo-box/llama.cpp:kv-disk-cache-haloboxCopy head branch name to clipboard