Chris Sanchez

The Effects of In-domain Corpus Size on pre-training BERT

2022 · arXiv (Cornell University)

Many prior language modeling efforts have shown that pre-training on an in-domain corpus can significantly improve performance on downstream domain-specific NLP tasks. However, the difficulties associated with collecting enough in-domain data might discourage researchers from …