A Parallel PSO Approach for Hyperparameter Optimization in Machine Learning Models

Abstract

Hyperparameter tuning is crucial for improving the performance of machine learning models, especially in high-dimensional and complex parameter spaces where traditional methods (Grid Search(GS) and Random Search (RS) fall short. This work introduces a parallelized Particle Swarm Optimization(P-PSO) approach for hyperparameter optimization, which is evaluated on three benchmark datasets (Iris, Breast Cancer, Red Wine Quality) across three models (Logistic Regression (LR), Random Forest (RF), and Support Vector Classifier (SVC)). Experimental results show that P-PSO achieves superior weighted F1-scores in most cases; for example, it reaches 0.96 on the Iris dataset across all models, 0.88 for RF on Breast Cancer, and 0.69 for RF on the particularly challenging Red Wine Quality dataset, outperforming other optimization techniques by margins of up to 0.02-0.05. Despite longer execution times, especially on complex models (up to 43 seconds for RF on Red Wine Quality), P-PSO offers more consistency and higher accuracy. These results confirm that P-PSO is an effective, scalable, and robust alternative for hyperparameter tuning, especially in cases where maximizing model performance rather than computational cost is prioritized.