Adaptive Privacy Budget Allocation in Federated Learning: A Multi-Agent Reinforcement Learning Approach

Federated learning is a popular distributed machine learning paradigm that keeps data locally at clients. To further enhance privacy protection, differential privacy techniques are incorporated in the federated learning framework. We can quantify the privacy budget (or privacy protection level) through differential privacy and allocate the budget to different communication rounds according to the composition property of differential privacy. Recent works have shown that suitably allocating budgets to different iterations can improve model performance. How to allocate privacy budgets in different communication rounds for different clients in the federated learning framework is a significant problem to study. The problem is challenging to solve due to the unknown relationship between noise levels and the model accuracy and the coupling property of the clients' decisions. In this paper, we propose a method based on multi-agent reinforcement learning to solve the privacy budget allocation problem, which maximizes the accuracy of the federated learning model given limited privacy budgets for the clients. The experiments show that our proposed method is better than the uniform allocation, arithmetic sequence allocation, and exponential allocation methods.