Xiao Dong Chen

TinyBERT: Distilling BERT for Natural Language Understanding

2020

Language model pre-training, such as BERT, has significantly improved the performances of many natural language processing tasks. However, pre-trained language models are usually computationally expensive, so it is difficult to efficiently execute them on resourcerestricted …