Picture for Giang Do

Giang Do

SimSMoE: Solving Representational Collapse via Similarity Measure

Add code
Jun 22, 2024
Viaarxiv icon

CompeteSMoE -- Effective Training of Sparse Mixture of Experts via Competition

Add code
Feb 04, 2024
Viaarxiv icon

HyperRouter: Towards Efficient Training and Inference of Sparse Mixture of Experts

Add code
Dec 12, 2023
Viaarxiv icon