Orhan Fırat

Leveraging Monolingual Data with Self-Supervision for Multilingual Neural Machine Translation

2020 · arXiv (Cornell University)

Over the last few years two promising research directions in low-resource neural machine translation (NMT) have emerged. The first focuses on utilizing high-resource languages to improve the quality of low-resource languages via multilingual NMT. The …

Multilingual Document-Level Translation Enables Zero-Shot Transfer From Sentences to Documents

2021 · arXiv (Cornell University)

Document-level neural machine translation (DocNMT) achieves coherent translations by incorporating cross-sentence context. However, for most language pairs there's a shortage of parallel documents, although parallel sentences are readily available. In this paper, we study whether …

On Using Monolingual Corpora in Neural Machine Translation

2015 · HAL (Le Centre pour la Communication Scientifique Directe)

Recent work on end-to-end neural network-based architectures for machine translation has shown promising results for En-Fr and En-De translation. Arguably, one of the major factors behind this success has been the availability of high quality …

Multi-Way, Multilingual Neural Machine Translation with a Shared Attention Mechanism

2016

We propose multi-way, multilingual neural machine translation. The proposed approach enables a single neural translation model to translate between multiple languages, with a number of parameters that grows only linearly with the number of languages. …

Montreal Neural Machine Translation Systems for WMT’15

2015

Neural machine translation (NMT) systems have recently achieved results comparable to the state of the art on a few translation tasks, including EnglishFrench and EnglishGerman. The main purpose of the Montreal Institute for Learning Algorithms …

Zero-Resource Translation with Multi-Lingual Neural Machine Translation

2016 · arXiv (Cornell University)

In this paper, we propose a novel finetuning algorithm for the recently introduced multi-way, mulitlingual neural machine translate that enables zero-resource machine translation. When used together with novel many-to-one translation strategies, we empirically show that …

Nematus: a Toolkit for Neural Machine Translation

2017

Rico Sennrich, Orhan Firat, Kyunghyun Cho, Alexandra Birch, Barry Haddow, Julian Hitschler, Marcin Junczys-Dowmunt, Samuel Läubli, Antonio Valerio Miceli Barone, Jozef Mokry, Maria Nădejde. Proceedings of the Software Demonstrations of the 15th Conference of the …

Zero-Shot Cross-lingual Classification Using Multilingual Neural Machine Translation

2018 · arXiv (Cornell University)

Transferring representations from large supervised tasks to downstream tasks has shown promising results in AI fields such as Computer Vision and Natural Language Processing (NLP). In parallel, the recent progress in Machine Translation (MT) has …

Massively Multilingual Neural Machine Translation

2019

Roee Aharoni, Melvin Johnson, Orhan Firat. Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers). 2019.

The Missing Ingredient in Zero-Shot Neural Machine Translation

2019 · arXiv (Cornell University)

Multilingual Neural Machine Translation (NMT) models are capable of translating between multiple source and target languages. Despite various approaches to train such models, they have difficulty with zero-shot translation: translating between language pairs that were …

Lingvo: a Modular and Scalable Framework for Sequence-to-Sequence Modeling

2019 · arXiv (Cornell University)

Lingvo is a Tensorflow framework offering a complete solution for collaborative deep learning research, with a particular focus towards sequence-to-sequence models. Lingvo models are composed of modular building blocks that are flexible and easily extensible, …

Massively Multilingual Neural Machine Translation in the Wild: Findings and Challenges

2019 · arXiv (Cornell University)

We introduce our efforts towards building a universal neural machine translation (NMT) system capable of translating between any language pair. We set a milestone towards this goal by building a single massively multilingual NMT model …

Simple, Scalable Adaptation for Neural Machine Translation

2019

Ankur Bapna, Orhan Firat. Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP). 2019.

XTREME: A Massively Multilingual Multi-task Benchmark for Evaluating Cross-lingual Generalization

2020 · arXiv (Cornell University)

Much recent progress in applications of machine learning models to NLP has been driven by benchmarks that evaluate models across a wide variety of tasks. However, these broad-coverage benchmarks have been mostly limited to English, …

Quality at a Glance: An Audit of Web-Crawled Multilingual Datasets

2022 · Transactions of the Association for Computational Linguistics

Abstract With the success of large-scale pre-training and multilingual modeling in Natural Language Processing (NLP), recent years have seen a proliferation of large, Web-mined text datasets covering hundreds of languages. We manually audit the quality …

PaLM: Scaling Language Modeling with Pathways

2022 · arXiv (Cornell University)

Large language models have been shown to achieve remarkable performance across a variety of natural language tasks using few-shot learning, which drastically reduces the number of task-specific training examples needed to adapt the model to …

Gemini: A Family of Highly Capable Multimodal Models

2023 · arXiv (Cornell University)

This report introduces a new family of multimodal models, Gemini, that exhibit remarkable capabilities across image, audio, video, and text understanding. The Gemini family consists of Ultra, Pro, and Nano sizes, suitable for applications ranging …