Exploring Robustness under New Adversarial Threats: A Comprehensive Analysis of Deep Neural Network Defenses

Although various methods have been proposed to improve the robustness of deep neural network models, evaluating these models in a fair and reasonable manner remains a challenge. Existing evaluation methods often consider only limited types of attacks, ignoring the generalization performance against new types of attacks, and fail to cover the most advanced defense models. To address these issues, we propose a unified framework to evaluate model robustness under different types of attacks. This framework integrates non-Lp and Lp attacks, and comprehensively evaluates the robustness of various advanced robust models on the CIFAR-10 and ImageNet subsets. Experimental results show that even the most advanced defense models exhibit vulnerabilities under certain new types of attacks, highlighting the importance of developing more comprehensive robustness benchmarks and providing guidance for the design of future robust deep learning models.