Federated learning on non-independent and identically distributed data

Federated Average algorithm (FEDAVG) is the preferred algorithm for federated learning (FL) because of its simplicity and low communication cost However, if all clients’s local data aren’t independent and equally distributed (that is, nonindependent and identically distributed), FedAvg will have the phenomenon of customer drift, which will lead to the slow convergence speed of the model, and then the efficient cooperative learning to realize the cooperative training of multiple clients will face the challenge of data heterogeneity and vulnerability to attack. This paper systematically summarizes three aspects: local model training improvement, server-side aggregation optimization, and personalized federated learning. The local model can be improved by adjusting the loss function and control variables. Servers can be optimized through asynchronous aggregation, hierarchical aggregation. Personalized federated learning can improve global model performance through both data-based and model-based approaches. This paper puts forward the future research direction of federated learning from all above aspects, and provides reference for the further research of non-IID in federated learning, so as to provide investigation and help for researchers in related fields.