Repository navigation

Feat/kv cache fp8 support - #1

Merged

luozixin2 merged 73 commits into

v0.0.1.0209from

feat/kv-cache-fp8-support

Feb 9, 2026

Merged

Feat/kv cache fp8 support#1
luozixin2 merged 73 commits into
v0.0.1.0209from
feat/kv-cache-fp8-support

Commits on Dec 15, 2025
- feat(kv-cache): add dtype protocol utilities for KV cache
  lzx
  committed
  e17bb0c
- feat(kv-cache): add multi-dtype support (BF16/FP16/FP32/FP8) to KV cache kernels
  lzx
  committed
  4f31a78
Commits on Dec 22, 2025
- fix(test): fix import path in test_dllm_kv_cache_store
  luozixin2
  committed
  4b2f61c
- test(kv-cache): add FP8 roundtrip tests for unified and distinct layouts
  luozixin2
  committed
  286bcf7
- fix(attention): optimize shared memory usage in prefix_prefill kernel
  luozixin2
  committed
  7589e92
Commits on Dec 24, 2025
- merge: merge origin/main into feat/kv-cache-fp8-support, preserve FP8 KV cache implementation
  luozixin2
  committed
  29e0bd2
- feat(kv-cache): add FP8 KV cache support in model_runner
  luozixin2
  committed
  fdd4bd5
- feat(attention): integrate FP8 KV cache support in attention layers
  luozixin2
  committed
  7496582
- fix(attention): fix parameter passing in store_kvcache calls
  luozixin2
  committed
  7080972
- test: add memory usage and speed comparison tests for FP8 KV cache
  luozixin2
  committed
  c26b135
- Merge pull request #13 from luozixin2/feat/kv-cache-fp8-support
  luozixin2
  authored
  835c47f
Commits on Dec 25, 2025
Commits on Dec 27, 2025
- fix: dllm_flash_attn_decode_kernel recompilation problem fixed
  drewjin
  committed
  191e706
Commits on Dec 28, 2025
Commits on Dec 29, 2025
- feat(strategy): create fast-dllm-v2 strategy
  drewjin
  committed
  535e296
- feat(fp8-kv): implement FP8 KV cache with Python dequantization
  luozixin2
  committed
  2828858
- update .gitignore
  drewjin
  committed
  90a518b
- feat(fp8-kv): implement FP8 distinct layout store and load
  luozixin2
  committed
  b97af94
- feat(sequence): add new sub-block statuses and attributes to FDV2SubBlock class
  drewjin
  committed
  714f915
- feat(attention): add wrapper functions for prefill/decode with quantization strategy support
  luozixin2
  committed
  746be44
- chore: update GitHub workflows to grant write permissions for issues and pull requests
  drewjin
  committed
  39c0d7e
- perf: optimize FP8 KV decode kernel using T.copy and scale fusion
  luozixin2
  committed
  6b1f450
Commits on Dec 30, 2025
- 重构量化模块架构并重命名FP8内核
  luozixin2
  committed
  6a951ea
- merge: merge origin/main into feat/kv-cache-fp8-support
  luozixin2
  committed
  820eac4
- Merge pull request #21 from luozixin2/feat/kv-cache-fp8-support
  luozixin2
  authored
  39cb15e
Commits on Dec 31, 2025
- feat: add Linear layer quantization strategy framework
  luozixin2
  committed
  65edadd
- feat: implement W8A16 Linear quantization strategy (int8 weight + bf16 activation)
  luozixin2
  committed
  fc32954
- perf: implement lazy cache for W8A16 Linear quantization strategy
  luozixin2
  committed
  266ea93
- feat: implement W8A16 TileLang kernel for Linear quantization
  luozixin2
  committed
  64e4347
- Merge branch 'main' into feat/fast-dllm-v2
  drewjin
  committed
  039693c
- feat: 为 test_text_generation.py 添加 warmup 机制和性能对比功能
  luozixin2
  committed
  ea47276
- feat: implement load-time quantization and memory-saving for W8A16 Linear layers
  luozixin2
  committed
  9ba300d
- Optimize W8A16 and W4A16 kernels: move per-channel scale from weight dequant to output scaling
  luozixin2
  committed
  ca3007c
Commits on Jan 1, 2026
- Improve W8A8/W4A8 quality by using FP16 scales instead of BF16
  luozixin2
  committed
  833b32c
Commits on Jan 5, 2026
Commits on Jan 6, 2026
- bugfix: try to fix profiler bug, upload and sync first
  drewjin
  committed
  7e65c0b
- Merge pull request #18 from drewjin/feat/enhance-strategy
  drewjin
  authored
  5b8352f
Commits on Jan 12, 2026
- Remove AttnQ quantization strategy support
  luozixin2
  committed
  c74b14b
- Merge remote-tracking branch 'fork/main' into feat/kv-cache-fp8-support
  luozixin2
  committed
  f8aa715
- Merge branch 'zhijie-group:feat/kv-cache-fp8-support' into feat/kv-cache-fp8-support
  luozixin2
  authored
  67686e0
- Merge branch 'zhijie-group:v0.0.1' into v0.0.1
  luozixin2
  authored
  0d9dd96
- Merge remote-tracking branch 'fork/v0.0.1' into feat/kv-cache-fp8-support
  luozixin2
  committed
  44fca07
Commits on Jan 13, 2026
- fix: 修复 FP8 KV cache RunningMax 策略中的 scale 更新逻辑
  luozixin2
  committed
  b4a4ed1
- chore: 移除 .cursor 目录并添加到 .gitignore
  luozixin2
  committed
  7b15d65
- Merge commit '67686e0' into feat/kv-cache-fp8-support
  luozixin2
  committed
  9015510
Commits on Jan 14, 2026
- feat: optimize W8A16 decode and FP8 KV varlen path
  luozixin2
  committed
  426b314
Commits on Jan 16, 2026
- feat: integrate Marlin/AllSpark INT8 W8A16 quantization strategy
  luozixin2
  committed
  dde9962
- Merge pull request #23 from luozixin2/feat/kv-cache-fp8-support
  luozixin2
  authored
  4a6e365
- feat: integrate Marlin/AllSpark INT8 W8A16 quantization strategy
  luozixin2
  committed
  3ec5e80
- Merge pull request #25 from luozixin2/feat/kv-cache-fp8-support
  luozixin2
  authored
  55b8b4d
Commits on Jan 18, 2026
- feat: 支持 GPTQ Marlin 和 AWQ Marlin 量化格式
  luozixin2
  committed
  a925717
- chore: 从仓库移除 benchmark_results
  luozixin2
  committed
  16d7892
- 升级 quantize_model.py 为真正的 GPTQ/AWQ 量化路径
  luozixin2
  committed
  a594135
- refactor: 优化代码结构和消除重复逻辑
  luozixin2
  committed
  8824ccd
Commits on Jan 24, 2026
- fix: 修正 bench 中 prefill/decode 吞吐量的平均值计算
  luozixin2
  committed
  23d377a
Commits on Jan 25, 2026
- perf: 优化量化 linear fast path 并移除 profiler 标注
  luozixin2
  committed
  896b8df
- refactor: remove CUDA Graph blockers and simplify linear quantization strategies
  luozixin2
  committed
  f6d0fa2
Commits on Jan 26, 2026
- perf: cache linear forward dispatch for CUDA Graph
  luozixin2
  committed
  7fba595
Commits on Jan 27, 2026
- Fix static+CUDA Graph mode and add benchmark configs
  luozixin2
  committed
  0d51145
Commits on Jan 28, 2026
- chore: 清理实验配置与环境变量依赖
  luozixin2
  committed
  8ea8717
- Merge pull request #26 from luozixin2/feat/kv-cache-fp8-support
  luozixin2
  authored
  600eb4c

Commits

Commits on Dec 15, 2025

Commits on Dec 22, 2025

Commits on Dec 24, 2025

Commits on Dec 25, 2025

Commits on Dec 27, 2025

Commits on Dec 28, 2025

Commits on Dec 29, 2025

Commits on Dec 30, 2025

Commits on Dec 31, 2025

Commits on Jan 1, 2026

Commits on Jan 5, 2026

Commits on Jan 6, 2026

Commits on Jan 12, 2026

Commits on Jan 13, 2026

Commits on Jan 14, 2026

Commits on Jan 16, 2026

Commits on Jan 18, 2026

Commits on Jan 24, 2026

Commits on Jan 25, 2026

Commits on Jan 26, 2026

Commits on Jan 27, 2026

Commits on Jan 28, 2026