Matthew Barnett

An Empirical Study of Scaling Laws for Transfer

2024 · arXiv (Cornell University)

We present a limited empirical study of scaling laws for transfer learning in transformer models. More specifically, we examine a scaling law that incorporates a "transfer gap" term, indicating the effectiveness of pre-training on one …