Picture for Alaaeldin Mohamed Elnouby Ali

Alaaeldin Mohamed Elnouby Ali

Parameters vs FLOPs: Scaling Laws for Optimal Sparsity for Mixture-of-Experts Language Models

Add code
Jan 21, 2025
Viaarxiv icon