Picture for Langxiang Hu

Langxiang Hu

Optimizing Speculative Decoding for Serving Large Language Models Using Goodput

Add code
Jun 20, 2024
Viaarxiv icon