James H. Cross

Lifting the Curse of Multilinguality by Pre-training Modular Transformers

2022 · arXiv (Cornell University)

Multilingual pre-trained models are known to suffer from the curse of multilinguality, which causes per-language performance to drop as they cover more languages. We address this issue by introducing language-specific modules, which allows us to …

No Language Left Behind: Scaling Human-Centered Machine Translation

2022 · arXiv (Cornell University)

Driven by the goal of eradicating language barriers on a global scale, machine translation has solidified itself as a key focus of artificial intelligence research today. However, such efforts have coalesced around a small subset …