feat(kvcache): store the KV cache as fp8 e4m3 codes (--kv-cache-dtype… - #354
Open
ArqAlice wants to merge 14 commits into
Open
feat(kvcache): store the KV cache as fp8 e4m3 codes (--kv-cache-dtype…#354ArqAlice wants to merge 14 commits into
ArqAlice wants to merge 14 commits into
Enhance your code review process with GitHub Actions
GitHub Actions make it easy to automate all your software workflows, now with world-class CI/CD.
Build, test, and deploy your code right from GitHub. Learn more about GitHub Actions.
Linux, macOS, Windows, and containers
Matrix builds
Any language
Live logs
Built-in secret store
