Leonie Kreis

Adaptive Step Sizes for Preconditioned Stochastic Gradient Descent

2023 · arXiv (Cornell University)

This paper proposes a novel approach to adaptive step sizes in stochastic gradient descent (SGD) by utilizing quantities that we have identified as numerically traceable -- the Lipschitz constant for gradients and a concept of …