PDP-FD: Federated Knowledge Distillation Based on Personalized Differential Privacy

Federated learning (FL) is a privacy-preserving distributed machine learning approach that enables training by exchanging model parameters without uploading local private data. However, the data heterogeneity across clients poses significant challenges in achieving personalized privacy protection. Existing methods often struggle to balance privacy and model utility, especially with inconsistent data distributions. Personalized differential privacy (PDP) is a commonly used technique to provide differential privacy (DP) by introducing varying levels of noise for each client. The noise level directly affects the model's utility, making it crucial to precisely determine the appropriate noise for each client. To address this challenge, we propose a federated knowledge distillation method based on PDP, named PDP-FD. PDP-FD dynamically adjusts the network architecture of both base and personalized layers to align with the data characteristics of different clients, enhancing model personalization. Moreover, it allocates an appropriate privacy budget to each client based on the similarity between local and global models, thereby meeting diverse privacy needs. Experimental results show that PDP-FD significantly outperforms existing FL methods in accuracy, effectively balancing privacy protection and model utility.