Janet B. Pierrehumbert

Temporal Adaptation of BERT and Performance on Downstream Document Classification: Insights from Social Media

2021

Language use differs between domains and even within a domain, language use changes over time. For pre-trained language models like BERT, domain adaptation through continued pre-training has been shown to improve performance on in-domain downstream …