![[2602.16167] Muon with Spectral Guidance: Efficient Optimization for Scientific Machine Learning](https://arxiv.org/static/browse/0.3.4/images/arxiv-logo-fb.png){kind=logo}
[2602.16167] Muon with Spectral Guidance: Efficient Optimization for Scientific Machine Learning
Summary
The paper introduces SpecMuon, a novel optimizer that enhances the Muon optimizer for scientific machine learning by addressing challenges in gradient optimization, particularly in physics-informed neural networks.
Why It Matters
This research is significant as it proposes a new optimization technique that improves convergence and stability in complex machine learning tasks, particularly those involving physical constraints. By enhancing the Muon optimizer, SpecMuon could lead to more efficient training of models used in scientific applications, which is crucial for advancements in fields like physics and engineering.
Key Takeaways
- SpecMuon integrates spectral guidance with the Muon optimizer to improve optimization stability.
- The method adapts step sizes based on global loss energy, enhancing convergence rates.
- Numerical experiments show SpecMuon outperforms traditional optimizers like Adam and AdamW.
- Theoretical properties of SpecMuon include energy dissipation and global convergence guarantees.
- This approach is particularly beneficial for physics-informed neural networks and related applications.
Computer Science > Machine Learning arXiv:2602.16167 (cs) [Submitted on 18 Feb 2026] Title:Muon with Spectral Guidance: Efficient Optimization for Scientific Machine Learning Authors:Binghang Lu, Jiahao Zhang, Guang Lin View a PDF of the paper titled Muon with Spectral Guidance: Efficient Optimization for Scientific Machine Learning, by Binghang Lu and 2 other authors View PDF HTML (experimental) Abstract:Physics-informed neural networks and neural operators often suffer from severe optimization difficulties caused by ill-conditioned gradients, multi-scale spectral behavior, and stiffness induced by physical constraints. Recently, the Muon optimizer has shown promise by performing orthogonalized updates in the singular-vector basis of the gradient, thereby improving geometric conditioning. However, its unit-singular-value updates may lead to overly aggressive steps and lack explicit stability guarantees when applied to physics-informed learning. In this work, we propose SpecMuon, a spectral-aware optimizer that integrates Muon's orthogonalized geometry with a mode-wise relaxed scalar auxiliary variable (RSAV) mechanism. By decomposing matrix-valued gradients into singular modes and applying RSAV updates individually along dominant spectral directions, SpecMuon adaptively regulates step sizes according to the global loss energy while preserving Muon's scale-balancing properties. This formulation interprets optimization as a multi-mode gradient flow and enables principled co...
