Deep Neural Networks for Network Intrusion Detection

Abstract

Networks have become an indispensable part of people's lives. With the rapid development of new technologies such as 5G and Internet of Things, people are increasingly dependent on networks, and the scale and complexity of networks are ever-growing. As a result, cyber threats are becoming more and more diverse, frequent and sophisticated, which imposes great threats to the massive networked society. The confidential information of the network users can be leaked; The integrity of data transferred over the network can be tampered; And the computing infrastructures connected to the network can be attacked. Therefore, network intrusion detection system (NIDS) plays a crucial role in offering the modern society a secure and reliable network communication environment.

Rule-based NIDSs are effective in identifying known cyber-attacks but ineffective for novel attacks, and hence are unable to cope with the ever-evolving threat landscape today. Machine learning (ML)-based NIDSs with intelligent and automated capabilities, on the other hand, can recognize both known and unknown attacks. Traditional ML-based designs achieve a high threat detection performance at the cost of a large number of false alarms, leading to alert fatigue. Advanced deep learning (DL)-based designs with deep neural networks can effectively mitigate this problem and accomplish better generalization capability than the traditional ML-based NIDSs. However, existing DL-based designs are not mature enough and there is still large room for improvement.

To tackle the above problems, in this thesis, we first propose a two-stage deep neural network architecture, DualNet, for network intrusion detection. DualNet is constructed with a general feature extraction stage and a crucial feature learning stage. It can effectively reuse the spatial-temporal features in accordance with their importance to facilitate the entire learning process and mitigate performance degradation problem occurred in deep learning. DualNet is evaluated on a traditional popular NSL-KDD dataset and a modern near-real-world UNSW-NB15 dataset, which shows a high detection accuracy that can be achieved by DualNet.

Based on DualNet, we then propose an enhanced design, EnsembleNet. EnsembleNet is a deep ensemble neural network model, which is built with a set of specially designed deep neural networks that are integrated by an aggregation algorithm. The model also has an alert-output enhancement design to facilitate security team's response to the intrusions and hence reduce security risks. EnsembleNet is evaluated on two modern datasets, a near-real-world UNSW-NB15 dataset and a more recent and comprehensive TON_IoT dataset, which shows that EnsembleNet has a high generalization capability.

Our evaluations on the UNSW-NB15 dataset that is close to the real-world network traffic demonstrate that DualNet and EnsembleNet outperform state-of-the-art ML-based designs by achieving higher threat detection performance while keeping lower false alarm rate, which also demonstrates that deep neural networks have great application potential in network intrusion detection.