Michael Lucas

Domain-Specific Language Model Pretraining for Biomedical Natural Language Processing

2021 · ACM Transactions on Computing for Healthcare

Pretraining large neural language models, such as BERT, has led to impressive gains on many natural language processing (NLP) tasks. However, most pretraining efforts focus on general domain corpora, such as newswire and Web. A …