Zhehui Chen

Toward Deeper Understanding of Nonconvex Stochastic Optimization with Momentum using Diffusion Approximations.

2018 · arXiv (Cornell University)

Momentum Stochastic Gradient Descent (MSGD) algorithm has been widely applied to many nonconvex optimization problems in machine learning. Popular examples include training deep neural networks, dimensionality reduction, and etc. Due to the lack of convexity …