An Orthogonally Equivariant Estimator of the Covariance Matrix in High Dimensions and for Small Sample Sizes

We introduce an estimation method of covariance matrices in a high-dimensional setting, i.e., when the dimension of the matrix, , is larger than the sample size . Specifically, we propose an orthogonally equivariant estimator. The eigenvectors of such estimator are the same as those of the sample covariance matrix. The eigenvalue estimates are obtained from an adjusted profile likelihood function derived by approximating the integral of the density function of the sample covariance matrix over its eigenvectors, which is a challenging problem in its own right. Exact solutions to the approximate likelihood equations are obtained and employed to construct estimates that involve a tuning parameter. Bootstrap and cross-validation based algorithms are proposed to choose this tuning parameter under various loss functions. Finally, comparisons with two well-known orthogonally equivariant estimators of the covariance matrix are given, which are based on Monte-Carlo risk estimates for simulated data and misclassification errors in real data analyses. In addition, Monte-Carlo risk estimates are also provided to compare our estimates of eigenvalues to those of a consistent estimator of population eigenvalues.