Edgar Dobriban

The Implicit Regularization of Stochastic Gradient Flow for Least Squares

2020 · arXiv (Cornell University)

We study the implicit regularization of mini-batch stochastic gradient descent, when applied to the fundamental problem of least squares regression. We leverage a continuous-time stochastic differential equation having the same moments as stochastic gradient descent, …