The increase in the use of the Internet and web services and the advent of the fifth generation of cellular network technology (5G) along with ever-growing Internet of Things (IoT) data traffic will grow global internet usage. To ensure the security of future networks, machine learning-based intrusion detection and prevention systems (IDPS) must be implemented to detect new attacks, and big data parallel processing tools can be used to handle a huge collection of training data in these systems. In this paper Apache Spark, a general-purpose and fast cluster computing platform is used for processing and training a large volume of network traffic feature data. In this work, the most important features of the CSE-CIC-IDS2018 dataset are used for constructing machine learning models and then the most popular machine learning approaches, namely Logistic Regression, Support Vector Machine (SVM), three different Decision Tree Classifiers, and Naive Bayes algorithm are used to train the model using up to eight number of worker nodes. Our Spark cluster contains seven machines acting as worker nodes and one machine is configured as both a master and a worker. We use the CSE-CIC-IDS2018 dataset to evaluate the overall performance of these algorithms on Botnet attacks and distributed hyperparameter tuning is used to find the best single decision tree parameters. We have achieved up to 100% accuracy using selected features by the learning method in our experiments
翻译:为确保未来网络的安全,必须采用基于机器学习的入侵探测和预防系统(IDPS)来检测新的袭击,并使用大数据平行处理工具来处理这些系统中的大量培训数据。在本论文Apache Spark中,一个通用和快速集束计算平台用于处理和培训大量网络通信特征数据。在这项工作中,CSE-CIC-IDS2018数据集的最重要功能被用于构建机器学习模型,然后是最受欢迎的机器学习方法,即物流回归、支持矢量机(SVM)、三个不同的决策树分类仪和Nive Bayes算法,用于培训模型,使用多达8个工人节点。我们的Spoint集包含7台机器,作为工人节点,一个机器被配置为主和工人。我们使用CSE-CIC-IDS20的最重要的特征来构建机器来构建机器学习模型,然后用我们100-CIC-IDS的精确度参数进行最佳的测试。我们使用CSE-ID18的模型来进行我们所选的学习的系统测试,通过Sybrodeal sal sal salation sal squt the supation the sal sal sal supal sal sal sal sal sal sal squt the the me