feat(kvcache): 8-bit KV cache (q8_0 / fp8_e4m3) behind --kv-cache-dtype - #103
Open
lucaspirola wants to merge 2 commits into
Open
feat(kvcache): 8-bit KV cache (q8_0 / fp8_e4m3) behind --kv-cache-dtype#103lucaspirola wants to merge 2 commits into
lucaspirola wants to merge 2 commits into
Enhance your code review process with GitHub Actions
GitHub Actions make it easy to automate all your software workflows, now with world-class CI/CD.
Build, test, and deploy your code right from GitHub. Learn more about GitHub Actions.
Linux, macOS, Windows, and containers
Matrix builds
Any language
Live logs
Built-in secret store
