Nicolas Flammarion

Harder, Better, Faster, Stronger Convergence Rates for Least-Squares Regression

2016 · arXiv (Cornell University)

We consider the optimization of a quadratic objective function whose gradients are only accessible through a stochastic oracle that returns the gradient at any given point plus a zero-mean finite variance random error. We present …

On Convergence-Diagnostic based Step Sizes for Stochastic Gradient Descent

2020 · arXiv (Cornell University)

Constant step-size Stochastic Gradient Descent exhibits two phases: a transient phase during which iterates make fast progress towards the optimum, followed by a stationary phase during which iterates oscillate around the optimal point. In this …