Shi-Shang Wang

Ordered Local Momentum for Asynchronous Distributed Learning Under Arbitrary Delays

2026 · Proceedings of the AAAI Conference on Artificial Intelligence

Momentum SGD (MSGD) serves as a foundational optimizer in training deep models due to momentum's key role in accelerating convergence and enhancing generalization. Meanwhile, asynchronous distributed learning is crucial for training large-scale deep models, especially …