Ethan Perez

Discovering Language Model Behaviors with Model-Written Evaluations

2022 · arXiv (Cornell University)

As language models (LMs) scale, they develop many novel behaviors, good and bad, exacerbating the need to evaluate how they behave. Prior work creates evaluations with crowdwork (which is time-consuming and expensive) or existing data …

Looking Inward: Language Models Can Learn About Themselves by Introspection

2024 · arXiv (Cornell University)

Humans acquire knowledge by observing the external world, but also by introspection. Introspection gives a person privileged access to their current state of mind (e.g., thoughts and feelings) that is not accessible to external observers. …

ELI5: Long Form Question Answering

2019

We introduce the first large-scale corpus for long-form question answering, a task requiring elaborate and in-depth answers to openended questions. The dataset comprises 270K threads from the Reddit forum "Explain Like I'm Five" (ELI5) where …

Affordance-Compiled Intelligence: Observable-Only Cognitive Impedance Matching for No-Meta LLM-Integrated Systems

2026 · arXiv (Cornell University)

Affordance-Compiled Intelligence develops Cognitive Impedance Matching Theory (CIMT), an observable-only and no-meta protected compiler theory for LLM-integrated systems. The paper studies how a fixed model-policy can exhibit different operational capability when the surrounding world is …

Case-based Reasoning for Natural Language Queries over Knowledge Bases

2021 · Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing

Rajarshi Das, Manzil Zaheer, Dung Thai, Ameya Godbole, Ethan Perez, Jay Yoon Lee, Lizhen Tan, Lazaros Polymenakos, Andrew McCallum. Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing. 2021.

True Few-Shot Learning with Language Models

2021 · arXiv (Cornell University)

Pretrained language models (LMs) perform well on many tasks even when learning from a few examples, but prior work uses many held-out examples to tune various aspects of learning, such as hyperparameters, training objectives, and …

Red Teaming Language Models with Language Models

2022

Ethan Perez, Saffron Huang, Francis Song, Trevor Cai, Roman Ring, John Aslanides, Amelia Glaese, Nat McAleese, Geoffrey Irving. Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing. 2022.