Towards Automated and Explainable Threat Hunting with Generative AI

This paper describes an attempt to automate threat hunting, taking cyber threat intelligence messages as input and generating queries to search logs for attack evidence using a popular query language, Kibana, used in Security Operations Centers (SOC). Our prototype implementation, AIThreatTrack, uses GPT-4 to extract actionable threat intelligence from real-time messages like X and Slack. The core idea is to explain the extracted intelligence in terms of MITRE ATT&CK TTPs using a knowledge graph containing "is-a" and "part-of" relationships (extracted using GPT-4) with the following benefits:(a) Significantly reduced hallucinations from 47% (GPT-4) to 1.5% using two orthogonal ways to cross-check answers. (b) Gaining analysts’ trust with explained results. (c) Using chain-of-knowledge prompting to significantly improve query generation accuracy. This approach supports expanding the scope of the knowledge graph to further improve query generation. Our approach significantly outperforms the Retrieval-Augmented Generation (RAG) approach and chain-of-thought reasoning LLM in reducing hallucinations.