Gabriel Ilharco

Fine-Tuning Pretrained Language Models: Weight Initializations, Data Orders, and Early Stopping

2020 · arXiv (Cornell University)

Fine-tuning pretrained contextual word embedding models to supervised downstream tasks has become commonplace in natural language processing. This process, however, is often brittle: even with the same hyperparameter values, distinct random seeds can lead to …