Towards an Image Utility Assessment Framework for Machine Perception

In real-world applications, images and videos used in computer vision algorithms are often distorted due, e.g., to compression and transmission. As a result, they may lose relevant information content, or they may deviate significantly from the original data distribution used to train the machine task, rendering the visual content practically useless with respect to its initial purpose. Evaluating the utility of an image for machine tasks has received little attention so far in the literature. This concept of utility is substantially different from the visual quality typically used in image/video compression, as the latter is related to the perception of the human visual system. In this paper, we propose a definition of utility as the degree of confidence by which a machine task is able to take a decision. In this context, we propose a full-reference utility loss measure: we assume that the decision on the pristine image is correct (reference), and we measure the utility loss as the confidence reduction in the decision due to a noisy input with respect to this reference. We apply this general definition on two specific tasks, classification and object detection, and we study practical solutions to predict utility, as well as the ability of our utility measure to generalize across tasks.