pd-disaggregation

Here are 5 public repositories matching this topic...

ome-projects / ome

Open Model Engine (OME) — Kubernetes operator for LLM serving, GPU scheduling, and model lifecycle management. Works with SGLang, vLLM, TensorRT-LLM, and Triton

k8s llama oracle-cloud model-serving model-as-a-service multi-node-kubernetes llm vllm llm-inference qwen deepseek sglang kimi-k2 pd-disaggregation

Updated Oct 2, 2026
Go

athrael-soju / Narwhal

Star

Adaptive Disaggregated Inference on a Role-free Fleet!

amd gpu arrow scheduling cuda inference nvidia decode rocm prefill kv-cache llm-serving vllm llm-inference prefix-caching nixl pd-disaggregation disaggregated-serving disaggregated-inference

Updated Oct 3, 2026
Python

yangjingo / nano-xpu

Star

A lightweight LLM inference runtime, evolving toward heterogeneous, stage-disaggregated serving.

lightweight cuda pytorch epd afd heterogeneous-computing kv-cache llm-inference distributed-inference qwen3 pd-disaggregation nano-vllm disaggregated-inference

Updated Jul 12, 2026
Python

Akhilesh-Chandewar / pdserve

Star

Prefill/Decode-disaggregated LLM serving library in Go — KV-cache connectors + SLO-aware elastic GPU flipping; published benchmark suite (colocated TPOT p99 inflates 133x under burst)

go benchmarking slo kv-cache vllm llm-inference sglang pd-disaggregation gpu-serving

Updated Sep 17, 2026
Go

hanglingmu0-web / llama-slot-state-pd

Star

RPC distributed prefill + slot-state PD disaggregation PoC for llama.cpp

rpc kv-cache llama-cpp local-llm gguf distributed-inference pd-disaggregation disaggregated-inference prefill-decode-disaggregation slot-state

Updated Sep 7, 2026
PowerShell

Add this topic to your repo

To associate your repository with the pd-disaggregation topic, visit your repo's landing page and select "manage topics."

Learn more