Enhancing Zero-Day and Known Attacks Detection using Hybrid-Optimized Stacking Ensemble Classifier
Article Sidebar
Main Article Content
Abstract
Zero-day and malicious attacks are still a research challenge for host system security. Zero-day attacks targeting the host systems digital data. To address this problem, the proposed two-stage multi-layered framework detects both zero-day and known attacks. The first stage uses a correlation-based filtering approach. Followed by feature selection using the Boruta algorithm and then the autoencoder (BAE) anomaly detection module. The second stage develops a stacking ensemble classifier (SEC) and optimizes it using the Swarm-Bayesian Hybrid Optimization (SBHO) technique. The combined approach forms the BAE-SEC-SBHO model. The model has been evaluated on the UNSW Windows ToN 2020 dataset, which includes host behavioral features like process, disk, and memory activity. The suggested pipeline reduces false-negative rates and improves recall rates compared to traditional machine learning and deep learning models. The performance and robustness of the proposed model are validated through an ablation study and a Wilcoxon signed-rank test for host-based intrusion detection.
Article Details
This work is licensed under a Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 International License.
References
F. A. Sufi, “New Social Media-Driven Cyber Threat Intelligence,” Electronics, vol. 12, no. 5, p. 1242, 2023.
K. Birch, “Data Assets,” in Data Enclaves, Cham, Switzerland: Palgrave Macmillan, 2023, pp. 41–59.
P. Passeri, “Cyber Attacks Statistics,” HACKMAGEDDON, Feb. 7, 2025.
S. Morgan, “Cybercrime to Cost the World $10.5 Trillion Annually by 2025,” Cybercrime Magazine, vol. 13, no. 11, 2020.
S. Caner, N. Erdogmus and Y. M. Erten, “Performance Analysis and Feature Selection for Network-Based Intrusion Detection with Deep Learning,” Turkish Journal of Electrical Engineering & Computer Sciences, vol. 30, no. 3, pp. 629–643, 2022.
Y. Guo, “A Review of Machine Learning-Based Zero-Day Attack Detection: Challenges and Future Directions,” Computer Communications, vol. 198, pp. 175–185, 2023.
F. Alhaidari et al., “ZeVigilante: Detecting Zero-Day Malware Using Machine Learning and Sandboxing Analysis Techniques,” Computational Intelligence and Neuroscience, vol. 2022, Art. no. 1615528, 2022.
M. Sarhan et al., “From Zero-Shot Machine Learning to Zero-Day Attack Detection,” International Journal of Information Security, vol. 22, no. 4, pp. 947–959, 2023.
A. Tour´e et al., “A Framework for Detecting Zero-Day Exploits in Network Flows,” Computer Networks, vol. 248, Art. no. 110476, 2024.
A. Jain, R. Bagoria and P. Arora, “An Intelligent Zero-Day Attack Detection System Using Unsupervised Machine Learning for Enhancing Cybersecurity,” Knowledge-Based Systems, Art. no. 113833, 2025.
C. Liu, Z. Gu and J. Wang, “A Hybrid Intrusion Detection System Based on Scalable K-Means+ Random Forest and Deep Learning,” in IEEE Access, vol. 9, pp. 75729-75740, 2021.
R. A. Disha and S. Waheed, “Performance Analysis of Machine Learning Models for Intrusion Detection System Using Gini Impurity-Based Weighted Random Forest Feature Selection,” Cybersecurity, vol. 5, no. 1, Art. no. 1, 2022.
M. Bakro et al., “An Improved Design for a Cloud Intrusion Detection System Using Hybrid Feature Selection with ML Classifier,” IEEE Access, vol. 11, pp. 64228–64247, 2023.
A. Singh et al., “Intrusion Detection System: A Comparative Study of Machine Learning-Based IDS,” Journal of Database Management, vol. 35, no. 1, pp. 1–25, 2024.
A. L. Buczak and E. Guven, “A Survey of Data Mining and Machine Learning Methods for Cybersecurity Intrusion Detection,” IEEE Communications Surveys & Tutorials, vol. 18, no. 2, pp. 1153–1176, 2016.
A. Meshram and C. Haas, “Anomaly Detection in Industrial Networks Using Machine Learning: A Roadmap,” in Machine Learning for Cyber Physical Systems, Berlin, Germany: Springer, 2016, pp. 65–72.
Y. Han, Z. Hao, L. Cui, C. Wang and Y. Sang, “A Hybrid Monitoring Mechanism in Virtualized Environments,” 2016 IEEE Trustcom/BigDataSE/ISPA, Tianjin, China, pp. 1038-1045, 2016.
X. Huang et al., “A Method for Windows Malware Detection Based on Deep Learning,” Journal of Signal Processing Systems, vol. 93, no. 2, pp. 265–273, 2021.
S. -Y. Dai, Y. Fyodor, J. -S. Wu, C. -H. Lin, Y. Huang and S. -Y. Kuo, “Holography: A Hardware Virtualization Tool for Malware Analysis,” 2009 15th IEEE Pacific Rim International Symposium on Dependable Computing, Shanghai, China, pp. 263-268, 2009.
S. Dua and X. Du, Data Mining and Machine Learning in Cybersecurity. Boca Raton, FL, USA: CRC Press, 2016.
Z. Amiri et al., “Adventures in Data Analysis: A Systematic Review of Deep Learning Techniques for Pattern Recognition in Cyber-Physical-Social Systems,” Multimedia Tools and Applications, vol. 83, no. 8, pp. 22909–22973, 2024.
R. Ahmad et al., “Zero-Day Attack Detection: A Systematic Literature Review,” Artificial Intelligence Review, vol. 56, no. 10, pp. 10733–10811, 2023.
M. Sewak, S. K. Sahay and H. Rathore, “Deep Reinforcement Learning for Cybersecurity Threat Detection and Protection: A Review,” in Proceedings of the International Conference on Secure Knowledge Management, 2021, pp. 51–72.
U. S. Musa, M. Chhabra, A. Ali and M. Kaur, “Intrusion Detection System using Machine Learning Techniques: A Review,” 2020 International Conference on Smart Electronics and Communication (ICOSEC), Trichy, India, pp. 149-155, 2020.
F. O. Catak et al., “Deep Learning-Based Sequential Model for Malware Analysis Using Windows EXE API Calls,” PeerJ Computer Science, vol. 6, Art. no. e285, 2020.
P. P. Chandran et al., “Optimal Deep Belief Network Enabled Malware Detection and Classification Model,” Intelligent Automation & Soft Computing, vol. 35, no. 3, pp. 3349–3364, 2023.
M. Jaihuni et al., “A Novel Recurrent Neural Network Approach in Forecasting Short-Term Solar Irradiance,” ISA Transactions, vol. 121, pp. 63–74, 2022.
J. Korstanje, “RNNs Using SimpleRNN and GRU,” in Advanced Forecasting with Python, Berkeley, CA, USA: Apress, 2025, pp. 255–269.
L. B. De Amorim, G. D. Cavalcanti, and R. M. Cruz, “The Choice of Scaling Technique Matters for Classification Performance,” Applied Soft Computing, vol. 133, Art. no. 109924, 2023.
G. Manikandan et al., “Classification Models Combined with Boruta Feature Selection for Heart Disease Prediction,” Informatics in Medicine Unlocked, vol. 44, Art. no. 101442, 2024.
A. Legrand, H. Trannois and A. Cournier, “Use of Uncertainty with Autoencoder Neural Networks for Anomaly Detection,” 2019 IEEE Second International Conference on Artificial Intelligence and Knowledge Engineering (AIKE), Sardinia, Italy, pp. 32-35, 2019.
J. Kennedy and R. Eberhart, “Particle swarm optimization,” Proceedings of ICNN’95 International Conference on Neural Networks, Perth, WA, Australia, vol.4, pp. 1942-1948, 1995.
P. I. Frazier, “Bayesian Optimization,” in Recent Advances in Optimization and Modeling of Contemporary Problems, Catonsville, MD, USA: INFORMS, 2018, pp. 255–278.
N. Moustafa et al., “Federated TON IoT Windows Datasets for Evaluating AI-Based Security Applications,” in Proceedings of the IEEE TrustCom, 2020, pp. 848–855.
Q. Zhang et al., “Boosting Adversarial Attacks with Nadam Optimizer,” Electronics, vol. 12, no. 6, Art. no. 1464, 2023.
H. Hsu and P. A. Lachenbruch, “Paired tTest,” Wiley StatsRef: Statistics Reference Online, 2014.
