Qihang Lin

Normalized Gradient with Adaptive Stepsize Method for Deep Neural Network Training.

2017 · arXiv (Cornell University)

In this paper, we propose a generic and simple algorithmic framework for first order optimization. The framework essentially contains two consecutive steps in each iteration: 1) computing and normalizing the mini-batch stochastic gradient; 2) selecting …