Deep Neural Networks for Network Intrusion Detection

Download files
Access & Terms of Use
open access
Copyright: Yang, Shiyi
CC BY 4.0
Altmetric
Abstract
Networks have become an indispensable part of people's lives. With the rapid development of new technologies such as 5G and Internet of Things, people are increasingly dependent on networks, and the scale and complexity of networks are ever-growing. As a result, cyber threats are becoming more and more diverse, frequent and sophisticated, which imposes great threats to the massive networked society. The confidential information of the network users can be leaked; The integrity of data transferred over the network can be tampered; And the computing infrastructures connected to the network can be attacked. Therefore, network intrusion detection system (NIDS) plays a crucial role in offering the modern society a secure and reliable network communication environment. Rule-based NIDSs are effective in identifying known cyber-attacks but ineffective for novel attacks, and hence are unable to cope with the ever-evolving threat landscape today. Machine learning (ML)-based NIDSs with intelligent and automated capabilities, on the other hand, can recognize both known and unknown attacks. Traditional ML-based designs achieve a high threat detection performance at the cost of a large number of false alarms, leading to alert fatigue. Advanced deep learning (DL)-based designs with deep neural networks can effectively mitigate this problem and accomplish better generalization capability than the traditional ML-based NIDSs. However, existing DL-based designs are not mature enough and there is still large room for improvement. To tackle the above problems, in this thesis, we first propose a two-stage deep neural network architecture, DualNet, for network intrusion detection. DualNet is constructed with a general feature extraction stage and a crucial feature learning stage. It can effectively reuse the spatial-temporal features in accordance with their importance to facilitate the entire learning process and mitigate performance degradation problem occurred in deep learning. DualNet is evaluated on a traditional popular NSL-KDD dataset and a modern near-real-world UNSW-NB15 dataset, which shows a high detection accuracy that can be achieved by DualNet. Based on DualNet, we then propose an enhanced design, EnsembleNet. EnsembleNet is a deep ensemble neural network model, which is built with a set of specially designed deep neural networks that are integrated by an aggregation algorithm. The model also has an alert-output enhancement design to facilitate security team's response to the intrusions and hence reduce security risks. EnsembleNet is evaluated on two modern datasets, a near-real-world UNSW-NB15 dataset and a more recent and comprehensive TON_IoT dataset, which shows that EnsembleNet has a high generalization capability. Our evaluations on the UNSW-NB15 dataset that is close to the real-world network traffic demonstrate that DualNet and EnsembleNet outperform state-of-the-art ML-based designs by achieving higher threat detection performance while keeping lower false alarm rate, which also demonstrates that deep neural networks have great application potential in network intrusion detection.
Persistent link to this record
http://hdl.handle.net/1959.4/100189
DOI
https://doi.org/10.26190/unsworks/23881
Link to Publisher Version
Link to Open Access Version
Additional Link
Author(s)
Supervisor(s)
Creator(s)
Editor(s)
Translator(s)
Curator(s)
Designer(s)
Arranger(s)
Composer(s)
Recordist(s)
Conference Proceedings Editor(s)
Other Contributor(s)
Corporate/Industry Contributor(s)
Publication Year
2021
Resource Type
Thesis
Degree Type
Masters Thesis
UNSW Faculty
Files
download public version.pdf 3.25 MB Adobe Portable Document Format
Related dataset(s)
The UNSW-NB15 dataset
Dataset
The raw network packets of the UNSW-NB 15 dataset was created by the IXIA PerfectStorm tool in the Cyber Range Lab of the Australian Centre for Cyber Security (ACCS) for generating a hybrid of real modern normal activities and synthetic contemporary attack behaviours. Tcpdump tool is utilised to capture 100 GB of the raw traffic (e.g., Pcap files). This data set has nine types of attacks, namely, Fuzzers, Analysis, Backdoors, DoS, Exploits, Generic, Reconnaissance, Shellcode and Worms. The Argus, Bro-IDS tools are used and twelve algorithms are developed to generate totally 49 features with the class label.
New Generations of Internet of Things Datasets for Cybersecurity Applications based Machine Learning: TON_IoT Datasets
Dataset
Collecting and analysing heterogeneous data sources from the Internet of Things (IoT) and Industrial IoT (IIoT) are essential for training and validating the fidelity of cybersecurity applications-based machine learning. However, the analysis of those data sources is still a big challenge for reducing high dimensional space and selecting important features and observations from different data sources. The study proposes a new testbed for an IIoT network that was utilised for creating new datasets called TON_IoT that collected Telemetry data, Operating systems data and Network data. The testbed is deployed using multiple virtual machines including hosts of windows, Linux and Kali Linux operating systems to manage the interconnections between the three layers of IIoT, Cloud and Edge/Fog systems. The initial statistical evaluation of the datasets reveals their capability for evaluating cybersecurity applications such as intrusion detection, threat intelligence, adversarial machine learning and privacy-preserving models.
View full record Show statistics