Jiangang Bai

Syntax-BERT: Improving Pre-trained Transformers with Syntax Trees

2021 · arXiv (Cornell University)

Pre-trained language models like BERT achieve superior performances in various NLP tasks without explicit consideration of syntactic information. Meanwhile, syntactic information has been proved to be crucial for the success of NLP applications. However, how …