Mark Kurtz

oBERTa: Improving Sparse Transfer Learning via improved initialization, distillation, and pruning regimes

2023 · arXiv (Cornell University)

In this paper, we introduce the range of oBERTa language models, an easy-to-use set of language models which allows Natural Language Processing (NLP) practitioners to obtain between 3.8 and 24.3 times faster models without expertise …