John C. Duchi

Asynchronous stochastic convex optimization

2015 · arXiv (Cornell University)

We show that asymptotically, completely asynchronous stochastic gradient procedures achieve optimal (even to constant factors) convergence rates for the solution of convex optimization problems under nearly the same conditions required for asymptotic optimality of standard …

Multiclass Classification, Information, Divergence, and Surrogate Risk

2016 · arXiv (Cornell University)

We provide a unifying view of statistical information measures, multi-way Bayesian hypothesis testing, loss functions for multi-class classification problems, and multi-distribution $f$-divergences, elaborating equivalence results between all of these objects, and extending existing results for …

Stochastic Methods for Composite Optimization Problems

2017 · arXiv (Cornell University)

We consider minimization of stochastic functionals that are compositions of a (potentially) non-smooth convex function $h$ and smooth function $c$ and, more generally, stochastic weakly-convex functionals. We develop a family of stochastic methods---including a stochastic …

Adapting to Function Difficulty and Growth Conditions in Private Optimization

2021 · arXiv (Cornell University)

We develop algorithms for private stochastic convex optimization that adapt to the hardness of the specific function we wish to optimize. While previous work provide worst-case bounds for arbitrary convex functions, it is often the …

Fine-tuning in Federated Learning: a simple but tough-to-beat baseline.

2021 · arXiv (Cornell University)

We study the performance of federated learning algorithms and their variants in an asymptotic framework. Our starting point is the formulation of federated learning as a multi-criterion objective, where the goal is to minimize each …