Bryan McCann

GeDi: Generative Discriminator Guided Sequence Generation

2020 · arXiv (Cornell University)

While large-scale language models (LMs) are able to imitate the distribution of natural language well enough to generate realistic text, it is difficult to control which regions of the distribution they generate. This is especially …

Learned in Translation: Contextualized Word Vectors

2017 · arXiv (Cornell University)

Computer vision has benefited from initializing multiple deep layers with weights pretrained on large supervised training sets like ImageNet. Natural language processing (NLP) typically sees initialization of only the lowest layer of deep models with …

The Natural Language Decathlon: Multitask Learning as Question Answering

2018 · arXiv (Cornell University)

Deep learning has improved performance on many natural language processing (NLP) tasks individually. However, general NLP models cannot emerge within a paradigm that focuses on the particularities of a single metric, dataset, and task. We …

XLDA: Cross-Lingual Data Augmentation for Natural Language Inference and Question Answering

2019 · arXiv (Cornell University)

While natural language processing systems often focus on a single language, multilingual transfer learning has the potential to improve performance, especially for low-resource languages. We introduce XLDA, cross-lingual data augmentation, a method that replaces a …

CTRL: A Conditional Transformer Language Model for Controllable Generation

2019 · arXiv (Cornell University)

Large-scale language models show promising text generation capabilities, but users cannot easily control particular aspects of the generated text. We release CTRL, a 1.63 billion-parameter conditional transformer language model, trained to condition on control codes …

BERT is Not an Interlingua and the Bias of Tokenization

2019

Multilingual transfer learning can benefit both high-and low-resource languages, but the source of these improvements is not well understood. Cananical Correlation Analysis (CCA) of the internal representations of a pretrained, multilingual BERT model reveals that …

GeDi: Generative Discriminator Guided Sequence Generation

2021

Ben Krause, Akhilesh Deepak Gotmare, Bryan McCann, Nitish Shirish Keskar, Shafiq Joty, Richard Socher, Nazneen Fatema Rajani. Findings of the Association for Computational Linguistics: EMNLP 2021. 2021.