Rényi Distillation for Global Testing in Sparse Regression Problems

Many modern high-dimensional regression applications involve testing whether a large set of predictors jointly affect an outcome of interest. Methods that target sparse alternatives, such as Tukey's Higher Criticism, require that predictors follow an orthogonal design, and attempts to generalise these approaches to non-orthogonal designs have yet to yield powerful tests. We propose two new procedures. The first, \emph{R\'enyi Distillation} (RD), judiciously introduces noise into the observed outcomes vector assuming a sparse alternative to obtain mutually independent p-values, each measuring the significance of a given predictor. The second, the \emph{R\'enyi outlier test}, is a global test that achieves similar power to Higher Criticism but which can be calculated cheaply and exactly deep into the tail even when the number of hypotheses is very large. In simulation, we demonstrate how the combination of these two procedures yields a scalable approah for non-orthogonal designs that maintains power under sparse alternatives. We also briefly discuss the potential of RD for tackling sparse variable selection and prediction problems.