Skip to content

server: on a full KV cache, terminate one slot instead of every slot - #183

Open
danielhanchen wants to merge 3 commits into
masterfrom
fix/terminate-one-slot
Open

server: on a full KV cache, terminate one slot instead of every slot#183
danielhanchen wants to merge 3 commits into
masterfrom
fix/terminate-one-slot

server: note why slot_batched is safe to leave pointing at a terminat…

2948afd
Select commit
Loading
Failed to load commit list.

Workflow runs completed with no jobs