Alec Radford

Robust Speech Recognition via Large-Scale Weak Supervision

2022 · arXiv (Cornell University)

We study the capabilities of speech processing systems trained simply to predict large amounts of transcripts of audio on the internet. When scaled to 680,000 hours of multilingual and multitask supervision, the resulting models generalize …

Learning to Generate Reviews and Discovering Sentiment

2017 · arXiv (Cornell University)

We explore the properties of byte-level recurrent language models. When given sufficient amounts of capacity, training data, and compute time, the representations learned by these models include disentangled features corresponding to high-level concepts. Specifically, we …

Fine-Tuning Language Models from Human Preferences

2019 · arXiv (Cornell University)

Reward learning enables the application of reinforcement learning (RL) to tasks where reward is defined by human judgment, building a model of reward by asking humans questions. Most work on reward learning has used simulated …

Scaling Laws for Neural Language Models

2020 · arXiv (Cornell University)

This paper develops a transport-validity theory for agentic AI interventions that are first screened on small systems and later considered for frontier-scale deployment. Rather than predicting absolute frontier performance, it asks when a comparative gain …

Language Models are Few-Shot Learners

2020 · arXiv (Cornell University)

Recent work has demonstrated substantial gains on many NLP tasks and benchmarks by pre-training on a large corpus of text followed by fine-tuning on a specific task. While typically task-agnostic in architecture, this method still …