Da Yu

Differentially Private Fine-tuning of Language Models

2024 · Journal of Privacy and Confidentiality

We give simpler, sparser, and faster algorithms for differentially private fine-tuning of large-scale pre-trained language models, which achieve the state-of-the-art privacy versus utility tradeoffs on many standard NLP tasks. We propose a meta-framework for this …