Alexandre Ramé

Fishr: Invariant Gradient Variances for Out-of-Distribution\n Generalization

2021 · arXiv (Cornell University)

Learning robust models that generalize well under changes in the data\ndistribution is critical for real-world applications. To this end, there has\nbeen a growing surge of interest to learn simultaneously from multiple training\ndomains - while enforcing …

Model Ratatouille: Recycling Diverse Models for Out-of-Distribution Generalization

2022 · arXiv (Cornell University)

Foundation models are redefining how AI systems are built. Practitioners now follow a standard procedure to build their machine learning solutions: from a pre-trained foundation model, they fine-tune the weights on the target task of …

Rewarded soups: towards Pareto-optimal alignment by interpolating weights fine-tuned on diverse rewards

2023 · arXiv (Cornell University)

Foundation models are first pre-trained on vast unsupervised datasets and then fine-tuned on labeled data. Reinforcement learning, notably from human feedback (RLHF), can further align the network with the intended usage. Yet the imperfections in …