Urgent.News

What's breaking now, across thousands of outlets.

More in AI

Why MuonGS Beats AdamW and Muon in LLM Pretraining

Standard AdamW and vanilla Muon waste millions of GPU hours during LLM pretraining because uncontrolled weight-norm growth throttles the effective learning rate.

  • MuonGS outperforms AdamW and standard Muon in LLM pretraining
  • 15% reduction in compute for 400M dense and 1B MoE models
  • Gram-Schmidt projection controls weight exploration sublinearly

More from Thursday 29 October →