Yihe Dong

AdaRank: Disagreement Based Module Rank Prediction for Low-rank Adaptation

2024 · arXiv (Cornell University)

With the rise of language and multimodal models of ever-increasing size, pretraining a general-purpose foundational model and adapting it to downstream tasks has become common practice. To this end, adaptation efficiency can be a critical …