Ziye Guo

CADA: Communication-Adaptive Distributed Adam

2020 · arXiv (Cornell University)

Stochastic gradient descent (SGD) has taken the stage as the primary workhorse for large-scale machine learning. It is often used with its adaptive variants such as AdaGrad, Adam, and AMSGrad. This paper proposes an adaptive …