Xi Victoria Lin

Lifting the Curse of Multilinguality by Pre-training Modular Transformers

2022 · arXiv (Cornell University)

Multilingual pre-trained models are known to suffer from the curse of multilinguality, which causes per-language performance to drop as they cover more languages. We address this issue by introducing language-specific modules, which allows us to …

Analysing Off-The-Shelf Options for Question Answering with Portuguese FAQs

2022 · DROPS (Schloss Dagstuhl – Leibniz Center for Informatics)

Following the current interest in developing automatic question answering systems, we analyse alternative approaches for finding suitable answers from a list of Frequently Asked Questions (FAQs), in Portuguese. These rely on different technologies, some more …

OPT-IML: Scaling Language Model Instruction Meta Learning through the Lens of Generalization

2022 · arXiv (Cornell University)

Recent work has shown that fine-tuning large pre-trained language models on a collection of tasks described via instructions, a.k.a. instruction-tuning, improves their zero and few-shot generalization to unseen tasks. However, there is a limited understanding …