Xiaoqi Jiao

TinyBERT: Distilling BERT for Natural Language Understanding

2019 · arXiv (Cornell University)

Language model pre-training, such as BERT, has significantly improved the performances of many natural language processing tasks. However, pre-trained language models are usually computationally expensive, so it is difficult to efficiently execute them on resource-restricted …

TinyBERT: Distilling BERT for Natural Language Understanding

2020

Language model pre-training, such as BERT, has significantly improved the performances of many natural language processing tasks. However, pre-trained language models are usually computationally expensive, so it is difficult to efficiently execute them on resourcerestricted …