Hady Elsahar

On Reinforcement Learning and Distribution Matching for Fine-Tuning Language Models with no Catastrophic Forgetting

2022 · arXiv (Cornell University)

The availability of large pre-trained models is changing the landscape of Machine Learning research and practice, moving from a training-from-scratch to a fine-tuning paradigm. While in some applications the goal is to "nudge" the pre-trained …

Controlling Conditional Language Models without Catastrophic Forgetting

2021 · arXiv (Cornell University)

Machine learning is shifting towards general-purpose pretrained generative models, trained in a self-supervised manner on large amounts of data, which can then be applied to solve a large number of tasks. However, due to their …

T-REx: A Large Scale Alignment of Natural Language with Knowledge Base Triples

2018

Alignments between natural language and Knowledge Base (KB) triples are an essential prerequisite for training machine learning approaches employed in a variety of Natural Language Processing problems.These include Relation Extraction, KB Population, Question Answering and …

BLOOM: A 176B-Parameter Open-Access Multilingual Language Model

2022 · arXiv (Cornell University)

Large language models (LLMs) have been shown to be able to perform new tasks based on a few demonstrations or natural language instructions. While these capabilities have led to widespread adoption, most LLMs are developed …