Knowledge Sharing Enhanced Clustered Federated Learning for Heterogeneous Client Data

Clustered Federated Learning (CFL) is a machine learning paradigm that balances local and global training to address limited local data and reduce the negative effects of data heterogeneity on the global model. However, data heterogeneity and insufficient samples in small clusters remain key challenges in CFL, and existing methods often overlook the potential of cross-cluster knowledge sharing, hindering further performance gains. To address these challenges, a clustered federated learning framework with knowledge sharing, termed KS-CFL (Knowledge-Sharing Clustered Federated Learning), is proposed. First, to tackle client data heterogeneity, local model gradients are used to extract representative features, and a data-driven heterogeneity metric is designed to identify client differences and guide cluster formation. Second, a divisive iterative clustering method is introduced to adjust cluster structures during training dynamically, improving client clustering precision. Finally, a weight-sharing mechanism inspired by multitask learning is introduced to mitigate data scarcity in small clusters, enhance model performance, and accelerate convergence. Experimental results show that KS-CFL outperforms state-of-the-art CFL methods on the CIFAR-10 and FEMNIST datasets. In the FEMNIST-D2 scenario, KS-CFL improves accuracy by 3.69% over FLT and reduces variance, enhancing fairness. On CIFAR-10-D1 and CIFAR-10-D2, it achieves accuracy gains of 2.18% and 2.66%, respectively, with lower variance. These results highlight the effectiveness of KS-CFL in heterogeneous data environments.