Picture for Gabriel Teston

Gabriel Teston

Communication-Efficient Language Model Training Scales Reliably and Robustly: Scaling Laws for DiLoCo

Add code
Mar 12, 2025
Viaarxiv icon

Streaming DiLoCo with overlapping communication: Towards a Distributed Free Lunch

Add code
Jan 30, 2025
Viaarxiv icon