DFuzzer: Diversity-Driven Seed Queue Construction of Fuzzing for Deep Learning Models

In light of high-performance computer processing, massive datasets, and mighty algorithms, we are rapidly entering an age where the advanced deep learning (DL) capabilities are integrated into the contemporary software systems to fulfill critical tasks. Like “traditional” software, DL systems are not immune to faults, some of which may even cause catastrophic disasters. As a mainstream testing technique for DL systems, fuzzing attempts to generate a large amount of semirandom yet syntactically valid test cases, from which the so-called adversarial inputs can be found, indicating the detection of faults. Test cases in fuzzing are generated based on a seed queue, which is constructed by randomly selecting seeds from the existing test suite (that is, the set of test cases). In this article, we propose a diversity-driven approach, namely DFuzzer, for constructing seed queues in fuzzing. We particularly develop two algorithms, namely DFuzzer-IB and DFuzzer-FB, based on the information theory and deep features, respectively, to improve the diversity of seed queues. Experimental studies have been conducted to evaluate the proposed techniques based on five fuzzers, three datasets, and seven DL models. The experimental results show that both strategies can significantly improve the performance of the state-of-the-art fuzzers for DL, including DeepXplore, DLFuzz, Tensorfuzz, DeepHunter, and DeepSmartFuzzer, not only in terms of finding more adversarial inputs for triggering faults but also achieving higher coverage. Our article demonstrates that the improved diversity of seed queues and the resultant test cases can help achieve a high testing effectiveness of fuzzing.