Peter Richtárik

Better Theory for SGD in the Nonconvex World

2020 · arXiv (Cornell University)

Large-scale nonconvex optimization problems are ubiquitous in modern machine learning, and among practitioners interested in solving them, Stochastic Gradient Descent (SGD) reigns supreme. We revisit the analysis of SGD in the nonconvex setting and propose …

MARINA: Faster Non-Convex Distributed Learning with Compression

2021 · arXiv (Cornell University)

We develop and analyze MARINA: a new communication efficient method for non-convex distributed learning over heterogeneous datasets. MARINA employs a novel communication compression strategy based on the compression of gradient differences that is reminiscent of …

Federated Learning with Regularized Client Participation

2023 · arXiv (Cornell University)

Federated Learning (FL) is a distributed machine learning approach where multiple clients work together to solve a machine learning task. One of the key challenges in FL is the issue of partial participation, which occurs …

MAST: Model-Agnostic Sparsified Training

2023 · arXiv (Cornell University)

We introduce a novel optimization problem formulation that departs from the conventional way of minimizing machine learning model loss as a black-box function. Unlike traditional formulations, the proposed approach explicitly incorporates an initially pre-trained model …