Hongchao Fang

CERT: Contrastive Self-supervised Learning for Language Understanding

2020

Pretrained language models such as BERT, GPT have shown great effectiveness in language understanding. The auxiliary predictive tasks in existing pretraining approaches are mostly defined on tokens, thus may not be able to capture sentence-level …