Àlex R. Atrio

Small Batch Sizes Improve Training of Low-Resource Neural MT

2022 · arXiv (Cornell University)

We study the role of an essential hyper-parameter that governs the training of Transformers for neural machine translation in a low-resource setting: the batch size. Using theoretical insights and experimental evidence, we argue against the …