A comprehensive intrusion detection framework using boosting algorithms

Abstract

Intrusion Detection Systems are one of the most effective technologies that protect systems against cyber-attacks. In this study, a new Comprehensive Cyber Security Intrusion Detection Dataset (CCiDD) was created. The CCiDD_A and CCiDD_B datasets are derived from the created dataset. Two datasets were compared with the NSL-KDD, UNSW-NB15 and CSE-CIC-IDS2018 datasets. In the study, the most optimal features for all datasets were determined by the Extra Tree algorithm and the new sub-datasets were classified by machine learning methods with default parameters. As a result of the classification, LGBM and XGBoost algorithms were selected as the most successful algorithms. Hyper parameter optimization was applied to LGBM and XGBoost algorithms to increase classification performance. LGBM classifier surpassed XGBoost classifier in terms of performance and processing time. LGBM algorithm achieved performance values of 99.84%, 98.02%, 99.94%, 95.68% and 99.98% for NSL-KDD, UNSW-NB15, CSE-CICIDS2018, CCiDD_A and CCiDD_B datasets, respectively. Since detection time of attacks is a critical issue, the LGBM classifier is recommended for attack detection in terms of time and performance.