One Explanation is Not Enough: Structured Attention Graphs for Image\n Classification

Attention maps are a popular way of explaining the decisions of convolutional\nnetworks for image classification. Typically, for each image of interest, a\nsingle attention map is produced, which assigns weights to pixels based on\ntheir importance to the classification. A single attention map, however,\nprovides an incomplete understanding since there are often many other maps that\nexplain a classification equally well. In this paper, we introduce structured\nattention graphs (SAGs), which compactly represent sets of attention maps for\nan image by capturing how different combinations of image regions impact a\nclassifier's confidence. We propose an approach to compute SAGs and a\nvisualization for SAGs so that deeper insight can be gained into a classifier's\ndecisions. We conduct a user study comparing the use of SAGs to traditional\nattention maps for answering counterfactual questions about image\nclassifications. Our results show that the users are more correct when\nanswering comparative counterfactual questions based on SAGs compared to the\nbaselines.\n