Piyush Sharma

ALBERT: A Lite BERT for Self-supervised Learning of Language Representations

2019 · arXiv (Cornell University)

Increasing model size when pretraining natural language representations often results in improved performance on downstream tasks. However, at some point further model increases become harder due to GPU/TPU memory limitations and longer training times. To …