Runyao Chen

On Batch Adaptive Training for Deep Learning: Lower Loss and Larger Step Size

2018

Mini-batch gradient descent and its variants are commonly used in deep learning. The principle of mini-batch gradient descent is to use noisy gradient calculated on a batch to estimate the real gradient, thus balancing the …