Knowledge Distillation in Federated Edge Learning: A Survey

The increasing demand for intelligent services coupled with privacy protection of mobile and Internet of Things (IoT) devices motivates the widespread adoption of Federated Edge Learning (FEL), which enables devices to collaboratively train on-device Machine Learning (ML) models coordinated by the edge server without sharing their private data. However, the intrinsic constraints of device hardware, user behaviors' variability, and the network infrastructure pose significant challenges to FEL, requiring its algorithm design to take into account resource constraints, the need for personalization models, and diverse network conditions. Fortunately, Knowledge Distillation (KD) emerges as an important strategy to address these impediments within FEL. In this paper, we investigate the works that KD applies to FEL, discuss the limitations and open problems of existing KD-based FEL approaches, and provide guidance for their real deployment.