Differential-KV (DKV) is a sparse KV-cache inference runtime designed for high-efficiency, memory-bounded long-context Large Language Model (LLM) inference across Apple Silicon (MLX) and CUDA GPUs.
Fused Triton kernels for Transformer inference: RMSNorm+RoPE, Gated MLP, FP8 GEMM — CPU-testable references, autotuning, and benchmarking
KV Memory Orchestrator — a browser-native research simulator for deterministic KV-cache residency, SRAM/HBM tiering, DMA scheduling, and memory-fabric orchestration in future transformer inference systems. Patent No. 202641062302, filed in India.
Run LLMs on everyday hardware: memory-smart MoE scheduling, KV paging, and the `ge` CLI for local GGUF inference. Pairs with Green Compress.
Add a description, image, and links to the transformer-inference topic page so that developers can more easily learn about it.
To associate your repository with the transformer-inference topic, visit your repo's landing page and select "manage topics."