Neil Houlsby

UL2: Unifying Language Learning Paradigms

2022 · arXiv (Cornell University)

Existing pre-trained models are generally geared towards a particular class of problems. To date, there seems to be still no consensus on what the right architecture and pre-training setup should be. This paper presents a …