Xuan, Yin

Linear Chain Transformation: Expanding Optimization Dynamics for Fine-Tuning Large Language Models

2024 · arXiv (Cornell University)

Fine-tuning large language models (LLMs) has become essential for adapting pretrained models to specific downstream tasks. In this paper, we propose Linear Chain Transformation (LinChain), a novel approach that introduces a sequence of linear transformations …