Sergey Ioffe

Batch Renormalization: Towards Reducing Minibatch Dependence in Batch-Normalized Models

2017 · arXiv (Cornell University)

Batch Normalization is quite effective at accelerating and improving the training of deep models. However, its effectiveness diminishes when the training minibatches are small, or do not consist of independent samples. We hypothesize that this …