Avi Caciularu

Cross-Document Language Modeling.

2021 · arXiv (Cornell University)

We introduce a new pretraining approach for language models that are geared to support multi-document NLP tasks. Our cross-document language model (CD-LM) improves masked language modeling for these tasks with two key ideas. First, we …