Oaken: Fast and Efficient LLM Serving with Online-Offline Hybrid KV Cache Quantization

Add code
Mar 24, 2025

Share this with someone who'll enjoy it:

View paper onarxiv icon

Share this with someone who'll enjoy it: