Akiyoshi Tomihari

Understanding Linear Probing then Fine-tuning Language Models from NTK Perspective

2024 · arXiv (Cornell University)

The two-stage fine-tuning (FT) method, linear probing (LP) then fine-tuning (LP-FT), outperforms linear probing and FT alone. This holds true for both in-distribution (ID) and out-of-distribution (OOD) data. One key reason for its success is …