Prioritizing Vulnerability Assessment Items Using LLM Based on IoT Device Documentations

With the rapid increase in demand for IoT devices, malicious attacks targeting vulnerabilities in IoT devices have been frequent in recent years. It is highly expected that the vulnerabilities can be removed from them through vulnerability assessment. However, the wide variety of IoT devices is not standardized, and it is difficult to set up vulnerability assessment items mechanically for those IoT devices, which causes a major obstacle to automate the vulnerability assessment for IoT devices. In this paper, we propose a method to prioritize vulnerability assessment items for every IoT device by effectively utilizing a large language model (LLM). The proposed method generates the answers that take into account the specifications of individual IoT devices using LLM by introducing Retrieval Augmented Generation (RAG), and determines how much suitable each vulnerability assessment item is for every IoT device by calculating the suitability using semantic entropy. Performing security tests based on the suitability must improve time efficiency while maintaining the coverage of the tests. Through the evaluation experiments, we obtained a suitability of vulnerability assessment items for the two types of IoT devices, which indicates 0.8 or higher in both the devices in terms of area under the curve (AUC).