Skip to content

[KDA] KDA MTP decode: recurrent + KVBuffer chunkwise verify + flush #96

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
Longxmas merged 19 commits into from
Jul 9, 2026
Merged

[KDA] KDA MTP decode: recurrent + KVBuffer chunkwise verify + flush #96

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
19 commits
Select commit Hold shift + click to select a range
1f2a76c
[KDA] KDA MTP decode recurrent
Longxmas Jun 16, 2026
7008d27
[KDA] KDA MTP decode kvbuffer chunkwise verify + flush
Longxmas Jun 16, 2026
c017db1
[KDA] MTP decode review fixes: per-request runtime accept-len, skip p…
Longxmas Jun 16, 2026
9bf0493
[KDA] KDA MTP decode kvbuffer cg verify: bank-conflict fix + 3xTF32
Longxmas Jun 22, 2026
cf3c1e7
[KDA] KDA MTP decode: lower_bound safe gate for vk/ws/kv/cg/tp gates
Longxmas Jun 24, 2026
5fbb826
[KDA] KDA MTP decode: dynamic-N batch dim + ruff lint fixes
Longxmas Jun 24, 2026
71d73f2
[chore] apply ruff format
Longxmas Jun 24, 2026
0da49c7
[KDA] KDA MTP decode kvbuffer: batched all-layers flush
Longxmas Jun 24, 2026
802981d
[KDA] kvbuffer verify/flush: overflow-safe decay-ratio form (softplus…
Longxmas Jul 2, 2026
dc2eded
[KDA] cg kvbuffer: 3xTF32 for the dsu=0 state-update GEMM
Longxmas Jul 2, 2026
004b770
[KDA] kvbuffer flush: dedicated BV selection (always smallest tile)
Longxmas Jul 2, 2026
ccb17a4
[KDA] MTP vk verify: accept K-contiguous strided q/k/v (dyn-stride)
Longxmas Jul 3, 2026
88b68b4
[kda-mtp] recurrent_ws restore + naming pass (recurrent / tensor_core…
Longxmas Jul 3, 2026
c6f941c
[kda-mtp] recurrent_ws ilp=2 (write-bound verify) + kvbuffer dispatch…
Longxmas Jul 3, 2026
d95fb16
[kda-mtp] recurrent_ws + kvbuffer verify: accept K-contiguous strided…
Longxmas Jul 5, 2026
5fec6eb
[kda-mtp] tests: cula.kda public API + recurrent_vk/recurrent_kv sele…
Longxmas Jul 6, 2026
ed259d5
[kda-mtp] docstrings: tidy kvbuffer numerical-form; drop recurrent or…
Longxmas Jul 6, 2026
9e69e4e
[kda-mtp] recurrent dispatch: route T==4 large-batch to recurrent_ws
Longxmas Jul 7, 2026
dcad512
Merge upstream main into feat/kda-mtp-verify-4ops
Longxmas Jul 8, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 3 additions & 1 deletion REPO_LAYOUT.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -34,8 +34,10 @@ cuLA/
β”‚ β”‚ β”‚ β”œβ”€β”€ fwd_o.py # output (chunk_gla_fwd_o)
β”‚ β”‚ β”‚ β”œβ”€β”€ bwd_wy_dqkg.py# backward wy/dqkg fused (used by chunk_bwd)
β”‚ β”‚ β”‚ └── cp/ # SM100 intracard-CP: chunk_delta_h, pre_scan, merge
β”‚ β”‚ β”œβ”€β”€ decode/ # single-token decode
β”‚ β”‚ β”œβ”€β”€ decode/ # single-token + MTP decode
β”‚ β”‚ β”‚ β”œβ”€β”€ cute.py # kda_decode / fused_sigmoid_gating_delta_rule_update (CuTe DSL)
β”‚ β”‚ β”‚ β”œβ”€β”€ mtp.py # kda_decode_mtp recurrent / recurrent_ws MTP verify (CuTe DSL)
β”‚ β”‚ β”‚ β”œβ”€β”€ mtp_kvbuffer.py # KVBuffer chunkwise MTP verify (shuffle / tensor_core) + flush
β”‚ β”‚ β”‚ └── reference_fla.py
β”‚ β”‚ └── experimental/sm100_fused/ # [exp] unwired fully-fused
β”‚ β”‚ β”œβ”€β”€ kda_fully_fused_wip.py # KDAChunkwise (~6k lines)
Expand Down
Loading