Aditya Siddhant

Leveraging Monolingual Data with Self-Supervision for Multilingual Neural Machine Translation

2020 · arXiv (Cornell University)

Over the last few years two promising research directions in low-resource neural machine translation (NMT) have emerged. The first focuses on utilizing high-resource languages to improve the quality of low-resource languages via multilingual NMT. The …

nmT5 -- Is parallel data still relevant for pre-training massively multilingual language models?

2021 · arXiv (Cornell University)

Recently, mT5 - a massively multilingual version of T5 - leveraged a unified text-to-text format to attain state-of-the-art results on a wide variety of multilingual NLP tasks. In this paper, we investigate the impact of …

XTREME: A Massively Multilingual Multi-task Benchmark for Evaluating Cross-lingual Generalization

2020 · arXiv (Cornell University)

Much recent progress in applications of machine learning models to NLP has been driven by benchmarks that evaluate models across a wide variety of tasks. However, these broad-coverage benchmarks have been mostly limited to English, …

mT5: A massively multilingual pre-trained text-to-text transformer

2020 · arXiv (Cornell University)

The recent "Text-to-Text Transfer Transformer" (T5) leveraged a unified text-to-text format and scale to attain state-of-the-art results on a wide variety of English-language NLP tasks. In this paper, we introduce mT5, a multilingual variant of …

mT5: A Massively Multilingual Pre-trained Text-to-Text Transformer

2021

Linting Xue, Noah Constant, Adam Roberts, Mihir Kale, Rami Al-Rfou, Aditya Siddhant, Aditya Barua, Colin Raffel. Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies. …