Parallel Fuzzy C-Means Text Clustering Algorithm Based on Improved Canopy

Fuzzy C-Means Clustering (FCM) algorithm is a flexible clustering algorithm, which can be used for large-scale data mining. However, FCM algorithm is sensitive to the selection of cluster centers and runs slowly when clustering large-scale text data. Therefore, a parallel fuzzy C-means clustering algorithm based on improved Canopy is proposed. Canopy algorithm needs to specify a reasonable threshold to calculate, and the concept of weight is introduced to improve the Canopy algorithm to improve the FCM clustering accuracy. Firstly, the density value of each sample point in the data set is calculated, and the maximum value is selected as the first clustering center. In the calculation of the remaining sample points in the weight, and then select the center point of other clusters. Finally, the parallel computing is carried out through the Spark framework to improve the efficiency of the algorithm. The experimental results show that the fuzzy C-means parallel text clustering algorithm based on improved Canopy improves the clustering accuracy by 10% -20% compared with FCM algorithm, and the overall performance of the algorithm has been improved.