Novel multi-cube extreme learning machine with sparse attention and ridge regression technique for robust machine learning

Abstract

This study introduces a learning framework designed to enhance the representational capacity and stability of single-layer feed-forward networks (SLFN) when modelling nonlinear and high-dimensional data. To this end, the proposed multi-cube unit with sparse attention and ridge regularisation (MCU-SAR) method integrates three complementary components: (i) a multi-cube unit (MCU) architecture that explicitly encodes higher-order feature interactions, (ii) a sparse attention mechanism that suppresses low-informative multiplicative terms, and (iii) a ridge-regularised extreme learning machine (ELM) output layer to improve generalisation. The proposed model is evaluated on 25 publicly available datasets, including 17 classification and 8 regression tasks, and benchmarked against 15 baseline methods comprising gradient-based optimisation techniques, support vector machines (SVM), and various ELM-based approaches. Performance comparisons are conducted using the Friedman test. MCU-SAR demonstrates consistently strong performance, ranking first on the majority of the 25 benchmark datasets and achieving competitive accuracy in classification as well as low error levels in regression tasks, with all results supported by statistically significant p-values. These results demonstrate that the proposed framework provides a scalable, generalisable, and computationally efficient solution for both classification and regression problems, offering robust performance on engineering-oriented real-world datasets.