Franck Mamalet

Turbo-Muon: Accelerating Orthogonality-Based Optimization with Pre-Conditioning

2025 · arXiv (Cornell University)

Orthogonality-based optimizers, such as Muon, have recently shown strong performance across large-scale training and community-driven efficiency challenges. However, these methods rely on a costly gradient orthogonalization step. Even efficient iterative approximations such as Newton-Schulz remain …