Semiparametrically efficient semi-supervised statistical inference

Summary We consider statistical inference under a semi-supervised setting where we have access to both labelled and unlabelled observations. We ask the question: under what circumstances, and by how much, can incorporating the unlabelled data improve upon inference using the labelled data? To answer this question, we investigate semi-supervised inference through the lens of semiparametric efficiency theory. For general pathwise differentiable targets under semiparametric models, we derive the efficiency lower bound for asymptotically linear semi-supervised estimators. Perhaps not surprisingly, this bound coincides with the results in classical missing data theory when ‘positivity’ holds, i.e., when the limiting proportion of labelled observations is strictly positive. However, unlike missing data theory, our theory can handle the boundary regime in which the proportion of labelled observations converges to zero, which is in fact the setting of primary interest in the context of semi-supervised learning. We further characterize a broad class of well-specified targets for which unlabelled data cannot yield efficiency gains. These results shed insight into the promise and limitations of the very active field of prediction-powered inference. We propose an efficient semi-supervised estimator that attains the efficiency bound, and a prediction-powered estimator that incorporates black-box predictions and is guaranteed to be no worse than the supervised baseline in asymptotic variance. We illustrate our theoretical results via simulation studies and a real-data application.