Junyuan Shang

ERNIE 3.0: Large-scale Knowledge Enhanced Pre-training for Language Understanding and Generation

2021 · arXiv (Cornell University)

Pre-trained models have achieved state-of-the-art results in various Natural Language Processing (NLP) tasks. Recent works such as T5 and GPT-3 have shown that scaling up pre-trained language models can improve their generalization abilities. Particularly, the …