Vinh Q. Tran

Confident Adaptive Language Modeling

2022 · arXiv (Cornell University)

Recent advances in Transformer-based large language models (LLMs) have led to significant performance improvements across many tasks. These gains come with a drastic increase in the models' size, potentially leading to slow and costly use …

UL2: Unifying Language Learning Paradigms

2022 · arXiv (Cornell University)

Existing pre-trained models are generally geared towards a particular class of problems. To date, there seems to be still no consensus on what the right architecture and pre-training setup should be. This paper presents a …