Tristan Naumann

Domain-Specific Language Model Pretraining for Biomedical Natural Language Processing

2021 · ACM Transactions on Computing for Healthcare

Pretraining large neural language models, such as BERT, has led to impressive gains on many natural language processing (NLP) tasks. However, most pretraining efforts focus on general domain corpora, such as newswire and Web. A …

Fine-Tuning Large Neural Language Models for Biomedical Natural Language Processing

2021 · arXiv (Cornell University)

Motivation: A perennial challenge for biomedical researchers and clinical practitioners is to stay abreast with the rapid growth of publications and medical notes. Natural language processing (NLP) has emerged as a promising direction for taming …

Publicly Available Clinical BERT Embeddings

2019 · arXiv (Cornell University)

Contextual word embedding models such as ELMo (Peters et al., 2018) and BERT (Devlin et al., 2018) have dramatically improved performance for many natural language processing (NLP) tasks in recent months. However, these models have …

Publicly Available Clinical

2019

Contextual word embedding models such as ELMo and BERT have dramatically improved performance for many natural language processing (NLP) tasks in recent months. However, these models have been minimally explored on specialty corpora, such as …