Ohad Shamir

On the iteration complexity of oblivious first-order optimization algorithms

2016 · International Conference on Machine Learning

We consider a broad class of first-order optimization algorithms which are oblivious, in the sense that their step sizes are scheduled regardless of the function under consideration, except for limited side-information such as smoothness or …

Random Shuffling Beats SGD Only After Many Epochs on Ill-Conditioned\n Problems

2021 · arXiv (Cornell University)

Recently, there has been much interest in studying the convergence rates of\nwithout-replacement SGD, and proving that it is faster than with-replacement\nSGD in the worst case. However, known lower bounds ignore the problem's\ngeometry, including its condition …