KVSharer: Efficient Inference via Layer-Wise Dissimilar KV Cache Sharing

Add code
Oct 24, 2024

Share this with someone who'll enjoy it:

View paper onarxiv icon

Share this with someone who'll enjoy it: