Optimisation of Feature Selection Using Stochastic Diffusion Search and Machine Learning Models

Abstract

The ever-growing field of Computational Intelligence (CI) offers an alternative to researchers seeking to solve complex and real-world problems. The reviewed literature revealed that the increasing dimensionality of datasets poses a significant challenge, which can severely impact the performance of predictive models by introducing noise and complexity, leading to overfitting and increased computational demands. This study presents an in-depth analysis and optimisation of feature selection through Stochastic Diffusion Search (SDS), targeting the curse of dimensionality in machine learning applications. Before applying the Stochastic Diffusion Search (SDS) algorithm, the dataset underwent an initial round of performance evaluation using various Machine Learning models. These ML models include Logistic Regression (LR), Decision Tree (DT), Random Forest (RF), Support Vector Machine (SVM), Neural Network (NN), Gradient Boosting (GB), Linear Discriminant Analysis (LDA), and Quadratic Discriminant Analysis (QDA). The initial assessment provided a baseline against which the efficacy of the SDS feature selection could be measured. The effectiveness of SDS in enhancing model performance is empirically evaluated using the sonar signal dataset. The results are illustrated in the Tables; however, the key results indicate a uniform improvement across various machine learning models when features are selected via SDS, with the Gradient Boosting Classifier showing the most significant enhancement in accuracy, from 86.54% to 90.38%. This uplift highlights the capability of the SDS to distil the most relevant features, thereby boosting predictive power while potentially reducing overfitting and computation time. These findings underscore the critical role of optimised feature selection in building efficient and robust predictive models, especially in high-dimensional data landscapes. The study concludes with insights into the balance between model accuracy and computational efficiency, providing a benchmark for further research in feature selection methodologies.