SPAM: Spike-Aware Adam with Momentum Reset for Stable LLM Training

Add code
Jan 12, 2025
Figure 1 for SPAM: Spike-Aware Adam with Momentum Reset for Stable LLM Training
Figure 2 for SPAM: Spike-Aware Adam with Momentum Reset for Stable LLM Training
Figure 3 for SPAM: Spike-Aware Adam with Momentum Reset for Stable LLM Training
Figure 4 for SPAM: Spike-Aware Adam with Momentum Reset for Stable LLM Training

Share this with someone who'll enjoy it:

View paper onarxiv icon

Share this with someone who'll enjoy it: