Thijs Vogels

Optimizer Benchmarking Needs to Account for Hyperparameter Tuning

2020 · Infoscience (Ecole Polytechnique Fédérale de Lausanne)

The performance of optimizers, particularly in deep learning, depends considerably on their chosen hyperparameter configuration. The efficacy of optimizers is often studied under near-optimal problem-specific hyperparameters, and finding these settings may be prohibitively costly for …

RelaySum for Decentralized Deep Learning on Heterogeneous Data

2021 · arXiv (Cornell University)

In decentralized machine learning, workers compute model updates on their local data. Because the workers only communicate with few neighbors without central coordination, these updates propagate progressively over the network. This paradigm enables distributed training …

MultiModN- Multimodal, Multi-Task, Interpretable Modular Networks

2023 · arXiv (Cornell University)

Predicting multiple real-world tasks in a single model often requires a particularly diverse feature space. Multimodal (MM) models aim to extract the synergistic predictive potential of multiple data types to create a shared feature space …