Noise-Adjusted Shrinkage Covariance Estimation in High Dimensions

Abstract

High-dimensional covariance estimation remains a fundamental challenge when the number of variables (p) substantially exceeds the sample size (n). In such settings, the sample covariance matrix is unstable, singular, and heavily contaminated by estimation noise. Although shrinkage estimators improve stability and thresholding methods promote sparsity, each approach alone may introduce bias or lose structural information. This study proposes a Noise-Adjusted Shrinkage Covariance (NASC) framework as a post-processing enhancement strategy for shrinkage-based covariance estimators. The framework first stabilizes the covariance structure through shrinkage toward a structured target, then suppresses noise-induced small covariance entries via thresholding, and finally applies a stabilization step to ensure positive definiteness of the resulting estimator. Sensitivity analyses were conducted to investigate the effects of the shrinkage and thresholding parameters, and the Monte Carlo simulations were subsequently performed using the best-performing parameter configuration. The simulation results showed that shrinkage alone may not sufficiently suppress entrywise noise, whereas NASC-adjusted estimators improved upon their corresponding shrinkage baselines in many scenarios, with the strongest gains observed for sparse covariance structures and for shrinkage estimators that do not explicitly suppress entrywise estimation noise. Improvements were more limited for highly optimized shrinkage estimators. Real-data analyses were conducted on the SRBCT and colon cancer benchmark datasets. On the SRBCT dataset, numerical stability and positive-definiteness properties were examined, while LOOCV-LDA classification performance without prior feature selection or dimensionality reduction was evaluated on the colon cancer dataset. The results suggest that NASC provides a computationally simple and numerically stable extension to classical shrinkage covariance estimation methods for high-dimensions.