Michael Auli

Neural Text Generation from Structured Data with Application to the Biography Domain

2016

This paper introduces a neural model for concept-to-text generation that scales to large, rich domains. It generates biographical sentences from fact tables on a new dataset of biographies from Wikipedia. This set is an order …

deltaBLEU: A Discriminative Metric for Generation Tasks with Intrinsically Diverse Targets

2015 · arXiv (Cornell University)

We introduce Discriminative BLEU (deltaBLEU), a novel metric for intrinsic evaluation of generated text in tasks that admit a diverse range of possible outputs. Reference strings are scored for quality by human raters on a …

Facebook FAIR’s WMT19 News Translation Task Submission

2019

This paper describes Facebook FAIR's submission to the WMT19 shared news translation task. We participate in four language directions, English German and English Russian in both directions. Following our submission from last year, our baseline …

Multilingual Speech Translation with Efficient Finetuning of Pretrained Models

2020 · arXiv (Cornell University)

We present a simple yet effective approach to build multilingual speech-to-text (ST) translation by efficient transfer learning from pretrained speech encoder and text decoder. Our key finding is that a minimalistic LNA (LayerNorm and Attention) …

A Neural Network Approach to Context-Sensitive Generation of Conversational Responses

2015

Alessandro Sordoni, Michel Galley, Michael Auli, Chris Brockett, Yangfeng Ji, Margaret Mitchell, Jian-Yun Nie, Jianfeng Gao, Bill Dolan. Proceedings of the 2015 Conference of the North American Chapter of the Association for Computational Linguistics: Human …

Sequence Level Training with Recurrent Neural Networks

2015 · arXiv (Cornell University)

Many natural language processing applications use language models to generate text. These models are typically trained to predict the next word in a sequence, given the previous words and some context such as an image. …

A Convolutional Encoder Model for Neural Machine Translation

2017

The prevalent approach to neural machine translation relies on bi-directional LSTMs to encode the source sentence. We present a faster and simpler architecture based on a succession of convolutional layers. This allows to encode the …

Language Modeling with Gated Convolutional Networks

2016 · arXiv (Cornell University)

The pre-dominant approach to language modeling to date is based on recurrent neural networks. Their success on this task is often linked to their ability to capture unbounded context. In this paper we develop a …

Convolutional Sequence to Sequence Learning

2017 · arXiv (Cornell University)

The prevalent approach to sequence to sequence learning maps an input sequence to a variable length output sequence via recurrent neural networks. We introduce an architecture based entirely on convolutional neural networks. Compared to recurrent …

Understanding Back-Translation at Scale

2018 · arXiv (Cornell University)

An effective method to improve neural machine translation with monolingual data is to augment the parallel training corpus with back-translations of target language sentences. This work broadens the understanding of back-translation and investigates a number …

Wizard of Wikipedia: Knowledge-Powered Conversational agents

2018 · arXiv (Cornell University)

In open-domain dialogue intelligent agents should exhibit the use of knowledge, however there are few convincing demonstrations of this to date. The most popular sequence to sequence models typically "generate and hope" generic utterances that …

Pay Less Attention with Lightweight and Dynamic Convolutions

2019 · arXiv (Cornell University)

Self-attention is a useful mechanism to build generative models for language and images. It determines the importance of context elements by comparing each element to the current time step. In this paper, we show that …

fairseq: A Fast, Extensible Toolkit for Sequence Modeling

2019

Myle Ott, Sergey Edunov, Alexei Baevski, Angela Fan, Sam Gross, Nathan Ng, David Grangier, Michael Auli. Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics (Demonstrations). 2019.

ELI5: Long Form Question Answering

2019

We introduce the first large-scale corpus for long-form question answering, a task requiring elaborate and in-depth answers to openended questions. The dataset comprises 270K threads from the Reddit forum "Explain Like I'm Five" (ELI5) where …

Controllable Abstractive Summarization

2018

Current models for document summarization disregard user preferences such as the desired length, style, the entities that the user might be interested in, or how much of the document the user has already read. We …

Sequence Level Training with Recurrent Neural Networks

2016 · International Conference on Learning Representations

Abstract: Many natural language processing applications use language models to generate text. These models are typically trained to predict the next word in a sequence, given the previous words and some context such as an …

Language modeling with gated convolutional networks

2017 · International Conference on Machine Learning

The pre-dominant approach to language modeling to date is based on recurrent neural networks. Their success on this task is often linked to their ability to capture unbounded context. In this paper we develop a …

vq-wav2vec: Self-Supervised Learning of Discrete Speech Representations

2019 · ArXiv.org

We propose vq-wav2vec to learn discrete representations of audio segments through a wav2vec-style self-supervised context prediction task. The algorithm uses either a gumbel softmax or online k-means clustering to quantize the dense representations. Discretization enables …

vq-wav2vec: Self-Supervised Learning of Discrete Speech Representations

2020 · arXiv (Cornell University)

We propose vq-wav2vec to learn discrete representations of audio segments through a wav2vec-style self-supervised context prediction task. The algorithm uses either a gumbel softmax or online k-means clustering to quantize the dense representations. Discretization enables …

In-Kernel Aggregation and Broadcast Acceleration for Distributed Communication

2020 · arXiv (Cornell University)

Broadcasting and aggregation dominate the communication overhead in distributed systems, from machine learning training to data analytics. Current acceleration approaches require specialized hardware (RDMA) or dedicated resources (DPDK), limiting their deployment in commodity clouds. However, …

Beyond English-Centric Multilingual Machine Translation

2020 · arXiv (Cornell University)

Existing work in translation demonstrated the potential of massively multilingual machine translation by training a single model able to translate between any pair of languages. However, much of this work is English-Centric by training only …

XLS-R: Self-supervised Cross-lingual Speech Representation Learning at Scale

2022 · Interspeech 2022

This paper presents XLS-R, a large-scale model for cross-lingual speech representation learning based on wav2vec 2.0.We train models with up to 2B parameters on nearly half a million hours of publicly available speech audio in …