EntropicFL: Efficient Federated Learning via Data Entropy and Model Divergence

Federated Learning (FL) is a strategy for training distributed learning models. This approach gives rise to significant challenges including the non-independent and identically distributed (non-IID.) characteristics inherent to the training data, which can wield influence over the comprehensive accuracy of the global model. Moreover, the collaborative involvement of multiple clients in training protocols frequently engenders increased communication overheads and resource management overheads. In this research, we present an innovative strategy to address the inherent challenges of federated learning, including the communication overhead, data heterogeneity, and privacy preservation concerns. Our proposed approach centers on the concept of adaptive client selection, comprising a two-step process: firstly, the identification of a subset of clients possessing pertinent and representative data for participation in model training, and secondly, the determination of whether to transmit the local updates from these selected clients to the central aggregation server. Our methodology leverages the metrics of data entropy and model divergence to guide this client selection process. By applying this approach, we effectively mitigate communication overhead without compromising the accuracy achieved in the federated learning process.