Ilya Sutskever

Robust Speech Recognition via Large-Scale Weak Supervision

2022 · arXiv (Cornell University)

We study the capabilities of speech processing systems trained simply to predict large amounts of transcripts of audio on the internet. When scaled to 680,000 hours of multilingual and multitask supervision, the resulting models generalize …

Addressing the Rare Word Problem in Neural Machine Translation

2015

Thang Luong, Ilya Sutskever, Quoc Le, Oriol Vinyals, Wojciech Zaremba. Proceedings of the 53rd Annual Meeting of the Association for Computational Linguistics and the 7th International Joint Conference on Natural Language Processing (Volume 1: Long …

Multi-task Sequence to Sequence Learning

2015 · arXiv (Cornell University)

Sequence to sequence learning has recently emerged as a new paradigm in supervised learning. To date, most of its applications focused on only one task and not much work explored this framework for multiple tasks. …

Learning to Generate Reviews and Discovering Sentiment

2017 · arXiv (Cornell University)

We explore the properties of byte-level recurrent language models. When given sufficient amounts of capacity, training data, and compute time, the representations learned by these models include disentangled features corresponding to high-level concepts. Specifically, we …

Language Models are Few-Shot Learners

2020 · arXiv (Cornell University)

Recent work has demonstrated substantial gains on many NLP tasks and benchmarks by pre-training on a large corpus of text followed by fine-tuning on a specific task. While typically task-agnostic in architecture, this method still …