Panos Toulis

Convergence diagnostics for stochastic gradient descent with constant step size

2017 · arXiv (Cornell University)

Many iterative procedures in stochastic optimization exhibit a transient phase followed by a stationary phase. During the transient phase the procedure converges towards a region of interest, and during the stationary phase the procedure oscillates …