High-dimensional covariance estimation by pairwise likelihood truncation

Abstract Pairwise likelihood is an approximation of the full likelihood function that facilitates the analysis of high-dimensional covariance models. By combining marginal bivariate likelihoods, it effectively simplifies high-dimensional dependencies, making the estimation process more manageable. We introduce estimation of sparse high-dimensional covariance matrices by maximizing a truncated version of the pairwise likelihood function, obtained by including pairwise terms corresponding to nonzero covariance elements. To achieve truncation, we propose a novel approach that minimizes the L  2-distance between pairwise and full likelihood scores, supplemented by an L  1-penalty to discourage the inclusion of uninformative terms. Unlike existing regularization methods, our criterion emphasizes the selection of entire pairwise likelihood objects instead of shrinking individual covariance parameters, thus preserving the unbiasedness of the pairwise likelihood estimating equations. The resulting pairwise likelihood estimator is consistent and converges to the oracle maximum likelihood estimator, which assumes prior knowledge of nonzero covariance entries, even as the data dimension increases exponentially with the sample size.