Picture for Hubertus Franke

Hubertus Franke

Efficient Interactive LLM Serving with Proxy Model-based Sequence Length Prediction

Add code
Apr 12, 2024
Viaarxiv icon