Jakob Uszkoreit

A Decomposable Attention Model for Natural Language Inference

2016 · arXiv (Cornell University)

We propose a simple neural architecture for natural language inference.Our approach uses attention to decompose the problem into subproblems that can be solved separately, thus making it trivially parallelizable.On the Stanford Natural Language Inference (SNLI) …

Neural Paraphrase Identification of Questions with Noisy Pretraining

2017

We present a solution to the problem of paraphrase identification of questions. We focus on a recent dataset of question pairs annotated with binary paraphrase labels and show that a variant of the decomposable attention …

Attention Is All You Need

2025

The dominant sequence transduction models are based on complex recurrent or convolutional neural networks in an encoder-decoder configuration. The best performing models also connect the encoder and decoder through an attention mechanism. We propose a …

Coarse-to-Fine Question Answering for Long Documents

2017

Eunsol Choi, Daniel Hewlett, Jakob Uszkoreit, Illia Polosukhin, Alexandre Lacoste, Jonathan Berant. Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers). 2017.

Fast Decoding in Sequence Models using Discrete Latent Variables

2018 · arXiv (Cornell University)

Autoregressive sequence models based on deep neural networks, such as RNNs, Wavenet and the Transformer attain state-of-the-art results on many tasks. However, they are difficult to parallelize and are thus slow at processing long sequences. …

Universal Transformers

2018 · arXiv (Cornell University)

Recurrent neural networks (RNNs) sequentially process data by updating their state with each new data point, and have long been the de facto choice for sequence modeling tasks. However, their inherently sequential computation makes them …

Natural Questions: A Benchmark for Question Answering Research

2019 · Transactions of the Association for Computational Linguistics

We present the Natural Questions corpus, a question answering data set. Questions consist of real anonymized, aggregated queries issued to the Google search engine. An annotator is presented with a question along with a Wikipedia …

Insertion Transformer: Flexible Sequence Generation via Insertion Operations

2019 · arXiv (Cornell University)

We present the Insertion Transformer, an iterative, partially autoregressive model for sequence generation based on insertion operations. Unlike typical autoregressive models which rely on a fixed, often left-to-right ordering of the output, our approach accommodates …

Transforming machine translation: a deep learning system reaches news translation quality comparable to human professionals

2020 · Nature Communications

The quality of human translation was long thought to be unattainable for computer translation systems. In this study, we present a deep-learning system, CUBBITT, which challenges this view. In a context-aware blind evaluation by human …