Convolutional long short-term memory for fileless malware detection

In cybersecurity, the rise of fileless malware poses a significant challenge to endpoint security. Traditional detection methods often fail against these sophisticated attacks, necessitating advanced techniques like deep learning models. This study highlights the limitations of Bi-Directional Long S...

Full description

Bibliographic Details
Main Authors: Kareegalan, Kunaprasan, Asmawi, Aziah, Abdullah, Mohd Taufik, Ninggal, Mohd Izuan Hafez, Abdullah, Muhammad Daniel Hafiz, Muhsen, Yousif Raad
Format: Article
Language:English
Published: Semarak Ilmu Publishing 2025
Online Access:http://psasir.upm.edu.my/id/eprint/117331/
http://psasir.upm.edu.my/id/eprint/117331/1/117331.pdf
_version_ 1848867219938738176
author Kareegalan, Kunaprasan
Asmawi, Aziah
Abdullah, Mohd Taufik
Ninggal, Mohd Izuan Hafez
Abdullah, Muhammad Daniel Hafiz
Muhsen, Yousif Raad
author_facet Kareegalan, Kunaprasan
Asmawi, Aziah
Abdullah, Mohd Taufik
Ninggal, Mohd Izuan Hafez
Abdullah, Muhammad Daniel Hafiz
Muhsen, Yousif Raad
author_sort Kareegalan, Kunaprasan
building UPM Institutional Repository
collection Online Access
description In cybersecurity, the rise of fileless malware poses a significant challenge to endpoint security. Traditional detection methods often fail against these sophisticated attacks, necessitating advanced techniques like deep learning models. This study highlights the limitations of Bi-Directional Long Short-Term Memory (BLSTM) models in dynamic malware analysis and proposes enhancements through Convolutional Long Short-Term Memory (ConvLSTM) architecture. BLSTM models process input sequences in forward and backward directions, combining the results into one output. While this dual-layer approach improves analysis, it is time-consuming, potentially increasing the risk of fileless malware attacks. A key limitation of BLSTM is the lack of parameter sharing between forward and backward directions. This reduces its ability to capture spatial and temporal features simultaneously, hindering effectiveness in detecting fileless malware. To address this, the ConvLSTM model consolidates feature extraction within a single LSTM cell layer. ConvLSTM breaks down samples into subsequences and uses timesteps for additional feature extraction, enabling spatial-temporal data analysis and improving malware prediction accuracy. The model was tested using a dynamic malware dataset. Unlike traditional LSTM, ConvLSTM integrates convolutional layers, allowing parameter sharing across both spatial and temporal dimensions. This reduces computational complexity and improves model performance in handling multidimensional data. The research re-simulated prior work with BLSTM using the same malware dataset. The Spyder app ran the event simulator, and the ConvLSTM model's results replaced BLSTM's using identical parameters. Time, accuracy, and loss were the main performance metrics. ConvLSTM outperformed BLSTM, achieving 98% detection accuracy compared to BLSTM's 90%. It also significantly reduced processing time, averaging 10 seconds, while BLSTM took 22 seconds. ConvLSTM experienced lower losses, averaging 10% per epoch versus BLSTM's 20%. In conclusion, ConvLSTM offers superior performance over BLSTM in fileless malware detection. Its enhanced computational efficiency and ability to quickly mitigate threats make it a robust solution for fortifying endpoint security against evolving cyber threats. ConvLSTM holds potential in strengthening defense mechanisms against sophisticated malware attacks, providing a proactive approach to safeguarding networks and data.
first_indexed 2025-11-15T14:33:02Z
format Article
id upm-117331
institution Universiti Putra Malaysia
institution_category Local University
language English
last_indexed 2025-11-15T14:33:02Z
publishDate 2025
publisher Semarak Ilmu Publishing
recordtype eprints
repository_type Digital Repository
spelling upm-1173312025-05-14T06:45:17Z http://psasir.upm.edu.my/id/eprint/117331/ Convolutional long short-term memory for fileless malware detection Kareegalan, Kunaprasan Asmawi, Aziah Abdullah, Mohd Taufik Ninggal, Mohd Izuan Hafez Abdullah, Muhammad Daniel Hafiz Muhsen, Yousif Raad In cybersecurity, the rise of fileless malware poses a significant challenge to endpoint security. Traditional detection methods often fail against these sophisticated attacks, necessitating advanced techniques like deep learning models. This study highlights the limitations of Bi-Directional Long Short-Term Memory (BLSTM) models in dynamic malware analysis and proposes enhancements through Convolutional Long Short-Term Memory (ConvLSTM) architecture. BLSTM models process input sequences in forward and backward directions, combining the results into one output. While this dual-layer approach improves analysis, it is time-consuming, potentially increasing the risk of fileless malware attacks. A key limitation of BLSTM is the lack of parameter sharing between forward and backward directions. This reduces its ability to capture spatial and temporal features simultaneously, hindering effectiveness in detecting fileless malware. To address this, the ConvLSTM model consolidates feature extraction within a single LSTM cell layer. ConvLSTM breaks down samples into subsequences and uses timesteps for additional feature extraction, enabling spatial-temporal data analysis and improving malware prediction accuracy. The model was tested using a dynamic malware dataset. Unlike traditional LSTM, ConvLSTM integrates convolutional layers, allowing parameter sharing across both spatial and temporal dimensions. This reduces computational complexity and improves model performance in handling multidimensional data. The research re-simulated prior work with BLSTM using the same malware dataset. The Spyder app ran the event simulator, and the ConvLSTM model's results replaced BLSTM's using identical parameters. Time, accuracy, and loss were the main performance metrics. ConvLSTM outperformed BLSTM, achieving 98% detection accuracy compared to BLSTM's 90%. It also significantly reduced processing time, averaging 10 seconds, while BLSTM took 22 seconds. ConvLSTM experienced lower losses, averaging 10% per epoch versus BLSTM's 20%. In conclusion, ConvLSTM offers superior performance over BLSTM in fileless malware detection. Its enhanced computational efficiency and ability to quickly mitigate threats make it a robust solution for fortifying endpoint security against evolving cyber threats. ConvLSTM holds potential in strengthening defense mechanisms against sophisticated malware attacks, providing a proactive approach to safeguarding networks and data. Semarak Ilmu Publishing 2025-03-18 Article PeerReviewed text en cc_by_nc_4 http://psasir.upm.edu.my/id/eprint/117331/1/117331.pdf Kareegalan, Kunaprasan and Asmawi, Aziah and Abdullah, Mohd Taufik and Ninggal, Mohd Izuan Hafez and Abdullah, Muhammad Daniel Hafiz and Muhsen, Yousif Raad (2025) Convolutional long short-term memory for fileless malware detection. Journal of Advanced Research in Applied Sciences and Engineering Technology, 64 (4). pp. 136-157. ISSN 2462-1943 https://semarakilmu.com.my/journals/index.php/applied_sciences_eng_tech/article/view/13042 10.37934/araset.64.4.136157
spellingShingle Kareegalan, Kunaprasan
Asmawi, Aziah
Abdullah, Mohd Taufik
Ninggal, Mohd Izuan Hafez
Abdullah, Muhammad Daniel Hafiz
Muhsen, Yousif Raad
Convolutional long short-term memory for fileless malware detection
title Convolutional long short-term memory for fileless malware detection
title_full Convolutional long short-term memory for fileless malware detection
title_fullStr Convolutional long short-term memory for fileless malware detection
title_full_unstemmed Convolutional long short-term memory for fileless malware detection
title_short Convolutional long short-term memory for fileless malware detection
title_sort convolutional long short-term memory for fileless malware detection
url http://psasir.upm.edu.my/id/eprint/117331/
http://psasir.upm.edu.my/id/eprint/117331/
http://psasir.upm.edu.my/id/eprint/117331/
http://psasir.upm.edu.my/id/eprint/117331/1/117331.pdf