Prabhu Teja Sivaprasad

Optimizer Benchmarking Needs to Account for Hyperparameter Tuning

2020 · Infoscience (Ecole Polytechnique Fédérale de Lausanne)

The performance of optimizers, particularly in deep learning, depends considerably on their chosen hyperparameter configuration. The efficacy of optimizers is often studied under near-optimal problem-specific hyperparameters, and finding these settings may be prohibitively costly for …