Brian O’Horo

OPT-IML: Scaling Language Model Instruction Meta Learning through the Lens of Generalization

2022 · arXiv (Cornell University)

Recent work has shown that fine-tuning large pre-trained language models on a collection of tasks described via instructions, a.k.a. instruction-tuning, improves their zero and few-shot generalization to unseen tasks. However, there is a limited understanding …