Optimal selection of a common subset of covariates for different regressions

Given a regression dataset of size $n$, most of the classical model selection literature treats the problem of selecting a subset of covariates to be used for prediction of future responses. In this paper we assume that a sample of $\cal J$ regression datasets of different sizes from a population of datasets having the same set of covariates and responses is observed. The goal is to select, for each $n$, a single subset of covariates to be used for prediction of future responses for any dataset of size $n$ from the population (which may or may not be in the sample of datasets). The regression coefficients used in the prediction are estimated using the $n$ observations consisting of covariates and responses in the sample for which prediction of future responses is to be done, and thus they differ across different samples. For example, if the response is a diagnosis, and the covariates are medical background variables and measurements, the goal is to select a standard set of measurements for different clinics, say, where each clinic may estimate and use its own coefficients for prediction (depending on local conditions, prevalence, etc.). The selected subset naturally depends on the sample size $n$, with a larger sample size allowing a more elaborate model. Since we consider prediction for any (or a randomly chosen) dataset in the population, it is natural to consider random covariates. If the population consists of datasets that are similar, our approach amounts to borrowing information, leading to a subset selection that is efficient for prediction. On the other hand, if the datasets are dissimilar, then our goal is to find a "compromise" subset of covariates for the different regressions.