Skip to content
#

prefix-sharing

Star

Here are 3 public repositories matching this topic...

JohnScheuer / pipeline-parallel-kv-bench

Star 1

Analytical benchmark connecting pipeline parallelism and KV cache management: bubble-time overlap for disaggregation, prefix reuse tax of PP stages, and tiering speedup across NVLink, InfiniBand, and PCIe

performance-engineering benchmark simulation inference systems infiniband disaggregation pcie nvlink analytical-model tiering pipeline-parallelism kv-cache llm llm-serving machine-learning-infrastructure prefix-sharing kv-cache-management bubble-time
  • Updated Jul 26, 2026
  • Python

JohnScheuer / sliding-window-attention-kv-bench

Star 0

Analytical benchmark for sliding window attention KV cache management: quality vs window size tradeoffs, SWA vs eviction comparison, prefix sharing interaction, and operational window recommendations across four attention distributions

performance-engineering benchmark simulation inference systems attention mistral eviction analytical-model kv-cache llm long-context llm-serving machine-learning-infrastructure sliding-window-attention prefix-sharing serving-systems kv-cache-management
  • Updated Jul 26, 2026
  • Python

JohnScheuer / tensor-parallel-kv-cache-bench

Star 0

Analytical benchmark connecting tensor parallelism and KV cache management in LLM serving: GQA sharding regimes, all-gather vs recompute tradeoffs, PCIe viability, and operational decision matrix across NVLink, InfiniBand, and PCIe

performance-engineering benchmark simulation inference systems attention infiniband disaggregation pcie nvlink analytical-model gqa kv-cache tensor-parallelism llm llm-serving grouped-query-attention machine-learning-infrastructure prefix-sharing all-gather
  • Updated Jul 26, 2026
  • Python

Improve this page

Add a description, image, and links to the prefix-sharing topic page so that developers can more easily learn about it.

Curate this topic

Add this topic to your repo

To associate your repository with the prefix-sharing topic, visit your repo's landing page and select "manage topics."

Learn more