Matthieu Cord

Fishr: Invariant Gradient Variances for Out-of-Distribution\n Generalization

2021 · arXiv (Cornell University)

Learning robust models that generalize well under changes in the data\ndistribution is critical for real-world applications. To this end, there has\nbeen a growing surge of interest to learn simultaneously from multiple training\ndomains - while enforcing …

CoMFormer: Continual Learning in Semantic and Panoptic Segmentation

2022 · arXiv (Cornell University)

Continual learning for segmentation has recently seen increasing interest. However, all previous works focus on narrow semantic segmentation and disregard panoptic segmentation, an important task with real-world impacts. %a In this paper, we present the …

Model Ratatouille: Recycling Diverse Models for Out-of-Distribution Generalization

2022 · arXiv (Cornell University)

Foundation models are redefining how AI systems are built. Practitioners now follow a standard procedure to build their machine learning solutions: from a pre-trained foundation model, they fine-tune the weights on the target task of …

Rewarded soups: towards Pareto-optimal alignment by interpolating weights fine-tuned on diverse rewards

2023 · arXiv (Cornell University)

Foundation models are first pre-trained on vast unsupervised datasets and then fine-tuned on labeled data. Reinforcement learning, notably from human feedback (RLHF), can further align the network with the intended usage. Yet the imperfections in …

MOCA: Self-supervised Representation Learning by Predicting Masked Online Codebook Assignments

2023 · arXiv (Cornell University)

Self-supervised learning can be used for mitigating the greedy needs of Vision Transformer networks for very large fully-annotated datasets. Different classes of self-supervised learning offer representations with either good contextual reasoning properties, e.g., using masked …

Reliability in Semantic Segmentation: Can We Use Synthetic Data?

2023 · arXiv (Cornell University)

Assessing the robustness of perception models to covariate shifts and their ability to detect out-of-distribution (OOD) inputs is crucial for safety-critical applications such as autonomous vehicles. By nature of such applications, however, the relevant data …

A Concept-Based Explainability Framework for Large Multimodal Models

2024 · arXiv (Cornell University)

Large multimodal models (LMMs) combine unimodal encoders and large language models (LLMs) to perform multimodal tasks. Despite recent advancements towards the interpretability of these models, understanding internal representations of LMMs remains largely a mystery. In …