prefill-decode-disaggregation

Here are 5 public repositories matching this topic...

Intelligent-Microsystems-Lab / SplitZip

SplitZip: Ultra Fast Lossless KV Compression for Disaggregated LLM Serving

gpu triton mooncake lossless-compression kv-cache bf16 llm-serving llm-inference sglang kv-cache-compression disaggregated-inference neurips-2026 prefill-decode-disaggregation

Updated Sep 28, 2026
Python

smg-project / smg-docs

Star

(SMG) Shepherd Model Gateway official documentation, designed by studio-noiich.

gateway epd smg kimi vllm deepseek sglang tokenspeed prefill-decode-disaggregation

Updated Oct 3, 2026
Python

antonellof / cognitora-inference

Star

Orchestrate LLM inference across your entire fleet. Run vLLM, SGLang, TensorRT-LLM, llama.cpp, or MLX (Apple Silicon) as a coordinated multi-node cluster (KV-aware, disaggregation-ready, energy-aware) on bare metal, Kubernetes, or any major cloud, installed with a single curl line.

inference tensorrt kv-cache llm vllm sglang prefill-benchmarking llm-cluster prefill-decode-disaggregation

Updated Sep 23, 2026
Rust

hanglingmu0-web / llama-slot-state-pd

Star

RPC distributed prefill + slot-state PD disaggregation PoC for llama.cpp

rpc kv-cache llama-cpp local-llm gguf distributed-inference pd-disaggregation disaggregated-inference prefill-decode-disaggregation slot-state

Updated Sep 7, 2026
PowerShell

rebel-sbpark / rbln-pd-disaggregation

Star

Prefill/decode disaggregation for LLM serving on RBLN-CA25 NPUs (vllm_rbln optimum path) + heterogeneous GPU→NPU KV bridge (dlfloat16, verified cos=0.9997). Deployable kits with paste-in runbooks.

npu kv-cache vllm llm-inference rbln prefill-decode-disaggregation

Updated Jul 13, 2026

Add this topic to your repo

To associate your repository with the prefill-decode-disaggregation topic, visit your repo's landing page and select "manage topics."

Learn more