Jingfei Du

Pretrained Encyclopedia: Weakly Supervised Knowledge-Pretrained Language\n Model

2019 · arXiv (Cornell University)

Recent breakthroughs of pretrained language models have shown the\neffectiveness of self-supervised learning for a wide range of natural language\nprocessing (NLP) tasks. In addition to standard syntactic and semantic NLP\ntasks, pretrained models achieve strong improvements on …