Amir Gholami

Inefficiency of K-FAC for Large Batch Size Training

2019 · arXiv (Cornell University)

In stochastic optimization, using large batch sizes during training can leverage parallel resources to produce faster wall-clock training times per training epoch. However, for both training loss and testing error, recent results analyzing large batch …