Ari Holtzman

The Curious Case of Neural Text Degeneration

2019 · arXiv (Cornell University)

Despite considerable advancements with deep neural language models, the enigma of neural text degeneration persists when these models are tested as text generators. The counter-intuitive empirical observation is that even though the use of likelihood …

HellaSwag: Can a Machine Really Finish Your Sentence?

2019

introduced a new task of commonsense natural language inference: given an event description such as "A woman sits at a piano," a machine must select the most likely followup: "She sets her fingers on the …

The Curious Case of Neural Text Degeneration

2020 · arXiv (Cornell University)

Despite considerable advances in neural language modeling, it remains an open question what the best decoding strategy is for text generation from a language model (e.g. to generate a story). The counter-intuitive empirical observation is …

Rethinking the Role of Demonstrations: What Makes In-Context Learning Work?

2022

Large language models (LMs) are able to in-context learn—perform a new task via inference alone by conditioning on a few input-label pairs (demonstrations) and making predictions for new inputs. However, there has been little understanding …