Barret Zoph

Multi-Source Neural Translation

2016

We build a multi-source machine translation model and train it to maximize the probability of a target English string given French and German sources. Using the neural encoderdecoder framework, we explore several combination methods and …

Emergent Abilities of Large Language Models

2022 · arXiv (Cornell University)

Scaling up language models has been shown to predictably improve performance and sample efficiency on a wide range of downstream tasks. This paper instead discusses an unpredictable phenomenon that we refer to as emergent abilities …

Transfer Learning for Low-Resource Neural Machine Translation

2016 · Digital Collections portal (Koç University)

The encoder-decoder framework for neural machine translation (NMT) has been shown effective in large data scenarios, but is much less effective for low-resource languages. We present a transfer learning method that significantly improves BLEU scores …

Neural Architecture Search with Reinforcement Learning

2016 · arXiv (Cornell University)

Neural networks are powerful and flexible models that work well for many difficult learning tasks in image, speech and natural language understanding. Despite their success, neural networks are still hard to design. In this paper, …

PaLM: Scaling Language Modeling with Pathways

2022 · arXiv (Cornell University)

Large language models have been shown to achieve remarkable performance across a variety of natural language tasks using few-shot learning, which drastically reduces the number of task-specific training examples needed to adapt the model to …

Scaling Instruction-Finetuned Language Models

2022 · arXiv (Cornell University)

Finetuning language models on a collection of datasets phrased as instructions has been shown to improve model performance and generalization to unseen tasks. In this paper we explore instruction finetuning with a particular focus on …