Entropy and Autoencoder-Based Outlier Detection in Mixed-Type Network Traffic Data

Mixed-type data containing categorical and numerical features are pervasive in real life, but very limited outlier detection methods are available for these data. Some existing methods handle mixed-type data by feature converting, whereas their performance is downgraded by information loss and noise caused by the transformation. Meanwhile, the existing general algorithms cannot combine the characteristics of outliers in specific fields, leading to an unsatisfying performance in actual scenarios, such as the field of network security. This paper proposes a novel Entropy and Autoencoder-based Outlier Detection in mixed-type network traffic data, termed EAOD, which combines characteristics of outliers in specific fields and machine learning models to detect outliers. EAOD utilizes the expert rules made by domain knowledge summarized based on characteristics of existing outliers to label known outlier data. It performs holoentropy and deep autoencoder for the category and numerical feature spaces, respectively, in unlabeled data to obtain outlier scores integrated to get the final outlier scores via a dynamic integration strategy. Especially in the numerical feature space, to fully mine known outlier behavior patterns, deep autoencoders of outlier and normal types are constructed separately to capture unknown outliers jointly. Experiments show that EAOD significantly outperforms eight state-of-the-art outlier detectors on seven real network traffic datasets.