Performance evaluation of search result diversification methods and their stability

Recently search result diversification has attracted a lot of attention in the information retrieval and Web search research community. Although quite a few result diversification algorithms have been proposed, it lacks in comparative study on those algorithms. In this paper, we compare four explicit result diversification algorithms. Three of them are representative explicit result diversification methods xQuAD, PM2 and IA-SELECT. The fourth, CombSumDiv, is presented in this paper. The experiments are carried out with 4 data sets used in the web track of TREC. The experimental results show that the proposed method CombSumDiv is as good as xQuAD, while PM2 and IA-SELECT are not as good as the other two. Stability of the four algorithms are also tested in a variety of situations. Experiments show that all four result diversification methods have very good stability when the search condition is not ideal.