Picture for Zhijie Sun

Zhijie Sun

LocMoE+: Enhanced Router with Token Feature Awareness for Efficient LLM Pre-Training

Add code
May 24, 2024
Viaarxiv icon

LocMoE: A Low-overhead MoE for Large Language Model Training

Add code
Jan 25, 2024
Viaarxiv icon