Abnormal event detection based on masked reconstruction and dual-channel adversarial prediction

Abnormal event detection in computer vision addresses the task of identifying events that deviate from expected behavior in video scenes. Issues, such as occlusion in crowded scenes, the powerful generalization capabilities of deep neural networks, and the heavy reliance on contextual information, make this task particularly challenging. To address these issues, we propose a cascaded form of abnormal detection framework that combines the paradigms of reconstruction and prediction in this paper. First, stochastic masking techniques are employed for image reconstruction to alleviate the overgeneralization of neural networks under abnormal conditions. Second, an innovative motion characterization of frame-difference streak streams is introduced to better characterize the motion of video frames in crowded scenes. Finally, a dual-channel autoencoder-based prediction network is introduced to jointly learn appearance and motion features. This network captures contextual information to better generate predictive features. Meanwhile, adversarial learning is introduced for abnormal inference to improve the detection performance. Experimental results on several benchmark datasets validate the effectiveness of our approach.