Higher-order least squares: assessing partial goodness of fit of linear regression

We introduce a simple diagnostic test for assessing the overall or partial goodness of fit of linear regression. We propose to evaluate the sensitivity of the regression coefficient with respect to changes of the marginal distribution of covariates by comparing the so-called higher-order least squares with the usual least squares estimates. In spite of its simplicity, this strategy is extremely general and powerful, including high-dimensional settings. Specifically, we show that it allows to distinguish between confounded and unconfounded predictor variables as well as determining ancestor variables in linear structural equation models assuming some non-Gaussianity. Thus, we provide a test for partial goodness of fit.