Suvrit Sra

Fast stochastic optimization on Riemannian manifolds.

2016 · arXiv (Cornell University)

We study optimization of finite sums of \emph{geodesically} smooth functions on Riemannian manifolds. Although variance reduction techniques for optimizing finite-sum problems have witnessed a huge surge of interest in recent years, all existing work is …

Fast Stochastic Methods for Nonsmooth Nonconvex Optimization

2016 · arXiv (Cornell University)

We analyze stochastic algorithms for optimizing nonconvex, nonsmooth finite-sum problems, where the nonconvex part is smooth and the nonsmooth part is convex. Surprisingly, unlike the smooth case, our knowledge of this fundamental problem is very …

Proximal stochastic methods for nonsmooth nonconvex finite-sum optimization

2016 · Neural Information Processing Systems

We analyze stochastic algorithms for optimizing nonconvex, nonsmooth finite-sum problems, where the nonsmooth part is convex. Surprisingly, unlike the smooth case, our knowledge of this fundamental problem is very limited. For example, it is not …

Global optimality conditions for deep neural networks

2017 · arXiv (Cornell University)

We study the error landscape of deep linear and nonlinear neural networks with the squared error loss. Minimizing the loss of a deep linear neural network is a nonconvex problem, and despite recent progress, our …

Time varying regression with hidden linear dynamics

2021 · arXiv (Cornell University)

We revisit a model for time-varying linear regression that assumes the unknown parameters evolve according to a linear dynamical system. Counterintuitively, we show that when the underlying dynamics are stable the parameters of this model …