Improving the Performance of OPTICS on Short Text Clustering by IsoKernel and UMAP

Clustering of short text streams has become significant due to the popularity of social media platforms. such as Twitter, Facebook, and Weibo. Clustering of short text can automatically detect new topics. Most existing approaches exploit dimensionality reduction and feature representation methods to enhance the clustering quality. However, it's a big challenge that how to improve the clustering performance under the aspects of large noise and high dimensionality of short text. OPTICS(Ordered Partitions Clustering Algorithm) algorithm is a density-based clustering algorithm recognized for its robust handling of noise. However, OPTICS encounters challenges when dealing with high-dimensional datasets. This paper proposes a novel short text clustering method that uses UMAP(U niform Manifold Approximation and Projection) and IsoKernel(Isolation Kernel) to improve the performance of OPTICS(Ordered Partitions Clustering Algorithm) in short text clustering tasks. The proposed method can overcome the problems of large noise and high dimensionality in short texts. The experimental results prove that proposed method is better than the baseline, which is evaluated on purity, AMI and NMI metrics.