Effect of LLM's Personality Traits on Query Generation

Large language models (LLMs) have demonstrated strong performance across various natural language processing tasks and are increasingly integrated into daily life. Just as personality traits are crucial in human communication, they could also play a significant role in the behavior of LLMs, for instance, in the context of Retrieval Augmented Generation. Previous studies have shown that Big Five personality traits could be applied to LLMs, but their specific effects on information retrieval tasks have not been sufficiently explored. This study aims to examine how personality traits assigned to LLM agents affect their query formulation behavior and search performance. We propose a method to accurately assign personality traits to LLM agents based on the Big Five theory and verify its accuracy using the IPIP-NEO-120 scale. We then design a query generation experiment using the NTCIR Ad-Hoc test collections and evaluate the search performance of queries generated by different LLM agents. The results show that our method successfully assigns all five personality traits to LLM agents as intended. Additionally, the query generation experiment suggests that the assigned traits did influence the length and vocabulary choices of generated queries. Finally, the retrieval effectiveness of the traits varied across test collections, showing a relative improvement ranging from -7.7% to +4.6%, but these differences were not statistically significant.