Rylan Schaeffer

Pretraining Scaling Laws for Generative Evaluations of Language Models

2025 · arXiv (Cornell University)

Neural scaling laws have driven the field's ever-expanding exponential growth in parameters, data and compute. While scaling behaviors for pretraining losses and discriminative benchmarks are well established, generative benchmarks such as mathematical problem-solving or software …

Are Emergent Abilities of Large Language Models a Mirage?

2023 · arXiv (Cornell University)

Recent work claims that large language models display emergent abilities, abilities not present in smaller-scale models that are present in larger-scale models. What makes emergent abilities intriguing is two-fold: their sharpness, transitioning seemingly instantaneously from …