Real‐time video anomaly detection for smart surveillance

Abstract Human monitoring of surveillance cameras for anomaly detection may be a monotonous task as it requires constant attention to judge if the captured activities are anomalous or suspicious. This paper exploits background subtraction (BS), convolutional autoencoder, and object detection for a fully automated surveillance system. BS was performed by modelling each pixel as a mixture of Gaussians (MoG) to concatenate only the higher‐order learning in the foreground. Next, the foreground objects are fed to the convolutional autoencoders to filter out abnormal events from normal ones and automatically identify signs of threat and violence in real time. Then, object detection is introduced on the entire scene and the region of interest is highlighted with a bounding box to minimize human intervention in video stream processing. At recognition time, the network generates an alarm for the presence of an anomaly to notify of the identification of potentially suspicious actions. Finally, the complete system is validated upon several benchmark datasets and proved to be robust for complex video anomaly detection. The (AUC) average area under the curve for the frame‐level evaluation for all benchmarks is 94.94%. The best improvement ratio of AUC between the proposed system and state‐of‐the‐art methods is 7.7%.