Detecting and mitigating botnet attacks using deep learning in software-defined networks

Software-Defined Networking (SDN) is a newly emerging network architecture separating control and data planes. It provides easy and flexible organization, management, and communication of complex or large-scale networks. Its programmable and centralized interfaces facilitate making complex and intel...

Full description

Bibliographic Details
Main Author: Muhammad Waqas, Nadeem
Format: Final Year Project / Dissertation / Thesis
Published: 2023
Subjects:
Online Access:http://eprints.utar.edu.my/6246/
http://eprints.utar.edu.my/6246/1/CCA_MWN_2023.pdf
_version_ 1848886626415017984
author Muhammad Waqas, Nadeem
author_facet Muhammad Waqas, Nadeem
author_sort Muhammad Waqas, Nadeem
building UTAR Institutional Repository
collection Online Access
description Software-Defined Networking (SDN) is a newly emerging network architecture separating control and data planes. It provides easy and flexible organization, management, and communication of complex or large-scale networks. Its programmable and centralized interfaces facilitate making complex and intelligent network decisions seamlessly and dynamically and can address the requirements of the data centers for managing the entire network. It also provides opportunities for individuals and businesses to build custom network applications based on their requirements and enhance their network services. Although SDN architecture offers high benefits, it introduced a new array of security and privacy challenges (i.e., single point of failure) that can preclude the wide adoption of SDNs. The SDN controller is a crucial element that attracts attackers to launch malicious attacks or activities on the controller (s) through OpenFlow switches. Distributed Denial of Service (DDoS) and botnet attacks are considered dangerous threats for networks such as IoT, SDNs, cloud computing, etc. If the attacker accesses the SDN controller, it can reroute the network traffic, causing severe damage to the whole network. So, Network Intrusion Detection Systems (NIDSs) have become important tools to protect networks against malicious attacks. Deep learning (DL)- based network applications are trending and have shown promising results in detecting and mitigating potential threats with fast response. In this research, we analyze and show the classification performance in terms of detecting and real-time performance of various DL methods based on Recurrent Neural Networks (RNNs), Convolution Neural Networks (CNNs), Multilayer Perceptron (MLP), Deep Neural Networks (DNNs), and Long Short Term Memory (LSTM) for botnet-based DDoS attacks in an SDN environment. A new simulation-based dataset is developed and used to train deep learning methods. We also used feature weighting and threshold tuning methods to derive the significant features required for detection. The simulation outcomes and measurements are verified using a simulation-based dataset and a real-time testbed environment. The aim of comparative analysis among the DL methods is to find the lightweight DL method with baseline hyperparameters, features and data that can be easily acquired to detect botnet-based DDoS attacks. The performance of the methods is evaluated using different metrics such as accuracy, detection rate, training and detection times, precision, F1 score, True Positive Rate (TPR), and False Positive Rate (FPR). The outcomes proved that the DL methods produced good results using optimal features. Finally, based on the simulation results, we observed that the CNN method outperforms using the simulated dataset and in real testbed settings. The detection rate of CNN reaches 97% for attack flows and 99% for normal flows. We also adopted graph theory and dynamic flow deletion-based mitigation strategy to protect the SDN environment against botnet attacks.
first_indexed 2025-11-15T19:41:29Z
format Final Year Project / Dissertation / Thesis
id utar-6246
institution Universiti Tunku Abdul Rahman
institution_category Local University
last_indexed 2025-11-15T19:41:29Z
publishDate 2023
recordtype eprints
repository_type Digital Repository
spelling utar-62462024-03-26T15:39:34Z Detecting and mitigating botnet attacks using deep learning in software-defined networks Muhammad Waqas, Nadeem H Social Sciences (General) HM Sociology T Technology (General) TA Engineering (General). Civil engineering (General) Software-Defined Networking (SDN) is a newly emerging network architecture separating control and data planes. It provides easy and flexible organization, management, and communication of complex or large-scale networks. Its programmable and centralized interfaces facilitate making complex and intelligent network decisions seamlessly and dynamically and can address the requirements of the data centers for managing the entire network. It also provides opportunities for individuals and businesses to build custom network applications based on their requirements and enhance their network services. Although SDN architecture offers high benefits, it introduced a new array of security and privacy challenges (i.e., single point of failure) that can preclude the wide adoption of SDNs. The SDN controller is a crucial element that attracts attackers to launch malicious attacks or activities on the controller (s) through OpenFlow switches. Distributed Denial of Service (DDoS) and botnet attacks are considered dangerous threats for networks such as IoT, SDNs, cloud computing, etc. If the attacker accesses the SDN controller, it can reroute the network traffic, causing severe damage to the whole network. So, Network Intrusion Detection Systems (NIDSs) have become important tools to protect networks against malicious attacks. Deep learning (DL)- based network applications are trending and have shown promising results in detecting and mitigating potential threats with fast response. In this research, we analyze and show the classification performance in terms of detecting and real-time performance of various DL methods based on Recurrent Neural Networks (RNNs), Convolution Neural Networks (CNNs), Multilayer Perceptron (MLP), Deep Neural Networks (DNNs), and Long Short Term Memory (LSTM) for botnet-based DDoS attacks in an SDN environment. A new simulation-based dataset is developed and used to train deep learning methods. We also used feature weighting and threshold tuning methods to derive the significant features required for detection. The simulation outcomes and measurements are verified using a simulation-based dataset and a real-time testbed environment. The aim of comparative analysis among the DL methods is to find the lightweight DL method with baseline hyperparameters, features and data that can be easily acquired to detect botnet-based DDoS attacks. The performance of the methods is evaluated using different metrics such as accuracy, detection rate, training and detection times, precision, F1 score, True Positive Rate (TPR), and False Positive Rate (FPR). The outcomes proved that the DL methods produced good results using optimal features. Finally, based on the simulation results, we observed that the CNN method outperforms using the simulated dataset and in real testbed settings. The detection rate of CNN reaches 97% for attack flows and 99% for normal flows. We also adopted graph theory and dynamic flow deletion-based mitigation strategy to protect the SDN environment against botnet attacks. 2023-09-26 Final Year Project / Dissertation / Thesis NonPeerReviewed application/pdf http://eprints.utar.edu.my/6246/1/CCA_MWN_2023.pdf Muhammad Waqas, Nadeem (2023) Detecting and mitigating botnet attacks using deep learning in software-defined networks. Master dissertation/thesis, UTAR. http://eprints.utar.edu.my/6246/
spellingShingle H Social Sciences (General)
HM Sociology
T Technology (General)
TA Engineering (General). Civil engineering (General)
Muhammad Waqas, Nadeem
Detecting and mitigating botnet attacks using deep learning in software-defined networks
title Detecting and mitigating botnet attacks using deep learning in software-defined networks
title_full Detecting and mitigating botnet attacks using deep learning in software-defined networks
title_fullStr Detecting and mitigating botnet attacks using deep learning in software-defined networks
title_full_unstemmed Detecting and mitigating botnet attacks using deep learning in software-defined networks
title_short Detecting and mitigating botnet attacks using deep learning in software-defined networks
title_sort detecting and mitigating botnet attacks using deep learning in software-defined networks
topic H Social Sciences (General)
HM Sociology
T Technology (General)
TA Engineering (General). Civil engineering (General)
url http://eprints.utar.edu.my/6246/
http://eprints.utar.edu.my/6246/1/CCA_MWN_2023.pdf