Chenhe Dong

EfficientBERT: Progressively Searching Multilayer Perceptron via Warm-up Knowledge Distillation

2021 · arXiv (Cornell University)

Pre-trained language models have shown remarkable results on various NLP tasks. Nevertheless, due to their bulky size and slow inference speed, it is hard to deploy them on edge devices. In this paper, we have …