Konstantin Mishchenko

Prodigy: An Expeditiously Adaptive Parameter-Free Learner

2023 · arXiv (Cornell University)

We consider the problem of estimating the learning rate in adaptive methods, such as AdaGrad and Adam. We propose Prodigy, an algorithm that provably estimates the distance to the solution $D$, which is needed to …