Event Detection through Lexical Chain Based Semantic Similarity Algorithm

Abstract Twitter is a platform where millions of people tend to tweet about new events happening in their lives. Celebrities tweeting about the new product endorsement, Politicians tweeting about their views towards a policy or people, Natural calamities occurring are tweeted instantly. Studying this data can provide us with useful information. In this paper, we have proposed an Event detection using a lexical chain based semantic similarity algorithm, for detecting Events from Twitter streams. Lexical chains have been used to preserve the Lexical cohesion in a text. The Twitter data set was collected using “tweepy” API, then pre-processing was done, steps like tokenization, stop word removal, and stemming is carried out and stored the tweets in a text file. Then lexical chains were built using the tweets in the file. The formation of the key graph, with each node as a lexical chain, was carried out. Then the clustering algorithm ‘SCAN’ was used to cluster the lexical chains, the formed clusters represent the keywords of an event. The last summarization step was carried out for each cluster representing an event.