Picture for Giang Do

Giang Do

SimSMoE: Solving Representational Collapse via Similarity Measure

Add code
Jun 22, 2024
Figure 1 for SimSMoE: Solving Representational Collapse via Similarity Measure
Figure 2 for SimSMoE: Solving Representational Collapse via Similarity Measure
Figure 3 for SimSMoE: Solving Representational Collapse via Similarity Measure
Figure 4 for SimSMoE: Solving Representational Collapse via Similarity Measure
Viaarxiv icon

CompeteSMoE -- Effective Training of Sparse Mixture of Experts via Competition

Add code
Feb 04, 2024
Figure 1 for CompeteSMoE -- Effective Training of Sparse Mixture of Experts via Competition
Figure 2 for CompeteSMoE -- Effective Training of Sparse Mixture of Experts via Competition
Figure 3 for CompeteSMoE -- Effective Training of Sparse Mixture of Experts via Competition
Figure 4 for CompeteSMoE -- Effective Training of Sparse Mixture of Experts via Competition
Viaarxiv icon

HyperRouter: Towards Efficient Training and Inference of Sparse Mixture of Experts

Add code
Dec 12, 2023
Viaarxiv icon