Dung Nguyen

CAMEx: Curvature-aware Merging of Experts

2025 · arXiv (Cornell University)

Existing methods for merging experts during model training and fine-tuning predominantly rely on Euclidean geometry, which assumes a flat parameter space. This assumption can limit the model's generalization ability, especially during the pre-training phase, where …