Yukun Li

ERNIE: Enhanced Representation through Knowledge Integration

2019 · arXiv (Cornell University)

We present a novel language representation model enhanced by knowledge called ERNIE (Enhanced Representation through kNowledge IntEgration). Inspired by the masking strategy of BERT, ERNIE is designed to learn language representation enhanced by knowledge masking …

ERNIE 2.0: A Continual Pre-training Framework for Language Understanding

2019 · arXiv (Cornell University)

Recently, pre-trained models have achieved state-of-the-art results in various language understanding tasks, which indicates that pre-training on large-scale corpora may play a crucial role in natural language processing. Current pre-training procedures usually focus on training …

ERNIE 2.0: A Continual Pre-Training Framework for Language Understanding

2020 · Proceedings of the AAAI Conference on Artificial Intelligence

Recently pre-trained models have achieved state-of-the-art results in various language understanding tasks. Current pre-training procedures usually focus on training the model with several simple tasks to grasp the co-occurrence of words or sentences. However, besides …