Variable Selection Method Based on Partial Least Squares Regression combined with Monte Carlo, Uninformative Variable Elimination and Genetic Algorithm

Abstract

Background:
Feature selection is essential for building accurate and interpretable predictive models, particularly in high-dimensional datasets where multicollinearity, noise, and redundant variables can severely degrade model performance. Existing single-stage or heuristic selection methods often struggle to balance prediction accuracy and model parsimony.

Objective:
The objective of this study is to develop a hybrid variable selection framework, termed MC-UVE-GA-PLS, that integrates monte Carlo Uninformative Variable Elimination (MC-UVE), Genetic Algorithm (GA) optimization, and Partial Least Squares (PLS) regression to identify compact and informative predictor subsets for robust predictive modeling.

Methods:
In the proposed approach, MC-UVE is first applied to assess variable stability using Monte Carlo cross-validation and eliminate uninformative predictors. The retained variables are then optimized using a GA to identify an optimal subset that minimizes prediction error. PLS regression is employed throughout the process as the base modeling technique to handle multicollinearity and determine the optimal number of latent components.

Results:
The proposed method was evaluated on ten real-world datasets from the UCI Machine Learning Repository. Compared with All-PLS and GA-PLS, MC-UVE-GA-PLS reduced the number of predictors by 24.2%–93.9%, while typically using fewer PLS components (2–3 versus 3–4). Across most datasets, the proposed method achieved lower RMSEP values, with relative improvements of up to 9.6%, and consistently higher or comparable predictive performance ( ​). Statistical tests based on repeated Monte Carlo cross-validation confirmed that the improvements in prediction accuracy were significant for the majority of datasets ( ).

Conclusion:
The MC-UVE-GA-PLS framework provides an effective and statistically robust solution for feature selection in high-dimensional regression problems. By combining stability-based filtering, evolutionary optimization, and latent variable modeling, the proposed method achieves superior predictive performance with substantially reduced model complexity, making it suitable for applications in chemometrics, bioinformatics, and econometrics.

Jagannath University Journal of Science, Volume 12, Number 1, Jun. 2025, pp. 87−98