Yafu Li

SEE: Continual Fine-tuning with Sequential Ensemble of Experts

2025 · arXiv (Cornell University)

Continual fine-tuning of large language models (LLMs) suffers from catastrophic forgetting. Rehearsal-based methods mitigate this problem by retaining a small set of old data. Nevertheless, they still suffer inevitable performance loss. Although training separate experts …