Shuohuan Wang
5 أوراق في مجموعة PaperMetrix
أوراق هذا المؤلف
-
ERNIE: Enhanced Representation through Knowledge Integration
2019 · arXiv (Cornell University)
We present a novel language representation model enhanced by knowledge called ERNIE (Enhanced Representation through kNowledge IntEgration). Inspired by the masking strategy of BERT, ERNIE is designed to learn language representation enhanced by knowledge masking …
-
ERNIE 2.0: A Continual Pre-training Framework for Language Understanding
2019 · arXiv (Cornell University)
Recently, pre-trained models have achieved state-of-the-art results in various language understanding tasks, which indicates that pre-training on large-scale corpora may play a crucial role in natural language processing. Current pre-training procedures usually focus on training …
-
ERNIE 2.0: A Continual Pre-Training Framework for Language Understanding
2020 · Proceedings of the AAAI Conference on Artificial Intelligence
Recently pre-trained models have achieved state-of-the-art results in various language understanding tasks. Current pre-training procedures usually focus on training the model with several simple tasks to grasp the co-occurrence of words or sentences. However, besides …
-
ERNIE-M: Enhanced Multilingual Representation by Aligning Cross-lingual Semantics with Monolingual Corpora
2021 · Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing
Recent studies have demonstrated that pretrained cross-lingual models achieve impressive performance in downstream cross-lingual tasks. This improvement benefits from learning a large amount of monolingual and parallel corpora. Although it is generally acknowledged that parallel …
-
ERNIE 3.0: Large-scale Knowledge Enhanced Pre-training for Language Understanding and Generation
2021 · arXiv (Cornell University)
Pre-trained models have achieved state-of-the-art results in various Natural Language Processing (NLP) tasks. Recent works such as T5 and GPT-3 have shown that scaling up pre-trained language models can improve their generalization abilities. Particularly, the …