Transfer Learning Model for Anomaly Detection in Data Streaming - Data Engineering Perspective

The main objective of this paper is to implement a transfer learning model for predicting anomalies in online streaming data. Streaming data is a continuous data generation and transmission model with a huge amount of data, enabling different kinds of vulnerable attacks in the network. It leads to negative impacts on the overall network performance. Several earlier methods have been proposed to improve anomaly detection accuracy in streaming data, whereas the false positive rate is high. This paper has aimed to increase the anomaly detection rate with a reduced false positive rate. Hence, it proposed a novel transfer learning method for designing an effective anomaly detection model in data streaming applications. It implements a long., short-term memory for managing the continuous generation and transfer of data called streaming data because it has multiple built-in features like forget gate., which operates the memory by eliminating unwanted and redundant data flows in the streaming process. The LSTM model is deployed in a kind of MANET called VANET, where it is applied to detect anomalies during vehicle communication. This paper provides high prediction accuracy since it integrates various data analytics tasks, like preprocessing, feature extraction, and classification, which feed quality data and perform fast analysis. The LSTM can detect anomalies, including DoS, DDoS, Sybil, Sinkhole, Wormhole, and blackhole. The simulation is carried out by implementing LSTM in Python and executed on a benchmark dataset to verify the efficacy of LSTM. The output shows that the model provides higher accuracy, low latency, and high throughput and is suitable for many real-time applications like IoT networks and cybersecurity.