Filip de Roos

A Probabilistically Motivated Learning Rate Adaptation for Stochastic Optimization

2021 · arXiv (Cornell University)

Machine learning practitioners invest significant manual and computational resources in finding suitable learning rates for optimization algorithms. We provide a probabilistic motivation, in terms of Gaussian inference, for popular stochastic first-order methods. As an important …