Om Thakkar

Training Production Language Models without Memorizing User Data

2020 · arXiv (Cornell University)

This paper presents the first consumer-scale next-word prediction (NWP) model trained with Federated Learning (FL) while leveraging the Differentially Private Federated Averaging (DP-FedAvg) technique. There has been prior work on building practical FL infrastructure, including …

Evading Curse of Dimensionality in Unconstrained Private GLMs via Private Gradient Descent

2020 · arXiv (Cornell University)

We revisit the well-studied problem of differentially private empirical risk minimization (ERM). We show that for unconstrained convex generalized linear models (GLMs), one can obtain an excess empirical risk of $\tilde O\left(\sqrt{\texttt{rank}}/εn\right)$, where ${\texttt{rank}}$ is …