Xuyi Chen

ERNIE: Enhanced Representation through Knowledge Integration

2019 · arXiv (Cornell University)

We present a novel language representation model enhanced by knowledge called ERNIE (Enhanced Representation through kNowledge IntEgration). Inspired by the masking strategy of BERT, ERNIE is designed to learn language representation enhanced by knowledge masking …

ERNIE 3.0: Large-scale Knowledge Enhanced Pre-training for Language Understanding and Generation

2021 · arXiv (Cornell University)

Pre-trained models have achieved state-of-the-art results in various Natural Language Processing (NLP) tasks. Recent works such as T5 and GPT-3 have shown that scaling up pre-trained language models can improve their generalization abilities. Particularly, the …