Ashok Cutkosky

Extreme Memorization via Scale of Initialization

2021 · arXiv (Cornell University)

We construct an experimental setup in which changing the scale of initialization strongly impacts the implicit regularization induced by SGD, interpolating from good generalization performance to completely memorizing the training set while making little progress …

Momentum Aggregation for Private Non-convex ERM

2022 · arXiv (Cornell University)

We introduce new algorithms and convergence guarantees for privacy-preserving non-convex Empirical Risk Minimization (ERM) on smooth $d$-dimensional objectives. We develop an improved sensitivity analysis of stochastic gradient descent on smooth objectives that exploits the recurrence …