Renqian Luo

Task-Agnostic and Adaptive-Size BERT Compression

2021

While pre-trained language models such as BERT and RoBERTa have achieved impressive results on various natural language processing tasks, they have huge numbers of parameters and suffer from huge computational and memory costs, which make …

Achieving Human Parity on Automatic Chinese to English News Translation

2018 · arXiv (Cornell University)

Machine translation has made rapid advances in recent years. Millions of people are using it today in online translation systems and mobile applications in order to communicate across language barriers. The question naturally arises whether …

BioGPT: generative pre-trained transformer for biomedical text generation and mining

2022 · Briefings in Bioinformatics

Pre-trained language models have attracted increasing attention in the biomedical domain, inspired by their great success in the general natural language domain. Among the two main branches of pre-trained language models in the general language …