Popular repositories Loading

1. Annotated-LLM-Runtime Annotated-LLM-Runtime Public

   From-scratch, heavily-annotated CUDA inference runtime for Qwen2.5-Coder-7B on H100 (sm_90). Custom INT4 packer, fused GEMV, paged KV, split-KV attention, CUDA graph decode — every hot path comment…

   Cuda 16 4

2. WarpGroup-backend WarpGroup-backend Public

   A high-performance C++ backend for extreme-context LLM inference. It replaces item-count batching with dynamic, VRAM-aware First-Fit Decreasing (FFD) bin packing. By using PyBind11 for async queuei…

   Python 5 1

3. swarmkv swarmkv Public

   C++ inference runtime for llama.cpp that shares a single document KV-cache prefill across multiple analytical branches via snapshot fan-out. Eliminates redundant GPU compute and dramatically reduce…

   C++ 3 1

4. sionna-munich-ai-ran sionna-munich-ai-ran Public

   GPU-accelerated Network Digital Twin for AI-RAN experiments in Munich. Powered by NVIDIA Sionna RT and PyTorch for ray-traced RF simulation. Includes real-world Munich simulation, Mitsuba 3 backend…

   Python 2 1

5. ILCP-for-Agents ILCP-for-Agents Public

   Inductive Latent Context Persistence (ILCP) for Agentic AI. This infrastructure persists, routes, and reuses LLM latent context across multi-agent DAGs. By eliminating redundant prefix-prefill comp…

   Python 2

6. VRAM-Conductor VRAM-Conductor Public

   C++17 + CUDA orchestrator for running multiple LLM agents on one constrained GPU. `lmxd` daemon does NVML-seeded admission control to stop llama.cpp OOM crashes; `LayerStreamer` + `PinnedHostPool` …

   C++ 2