Confounder importance learning for treatment effect inference

We address applied and computational issues for the problem of multiple treatment effect inference under many potential confounders. While there is abundant literature on the harmful effects of omitting relevant controls (under-selection), we show that over-selection can be comparably problematic, introducing substantial variance and a bias related to the non-random over-inclusion controls. We provide a novel empirical Bayes framework to mitigate both under-selection problems in standard high-dimensional methods and over-selection issues in recent proposals, by learning whether each control's inclusion should be encouraged or discouraged. We develop efficient gradient-based and Expectation-Propagation model-fitting algorithms to render the approach practical for a wide class of models. A motivating problem is to estimate the salary gap evolution in recent years in relation to potentially discriminatory characteristics such as gender, race, ethnicity and place of birth. We found that, albeit smaller, some wage differences remained for female and black individuals. A similar trend is observed when analyzing the joint contribution of these factors to deviations from the average salary. Our methodology also showed remarkable estimation robustness to the addition of spurious artificial controls, relative to existing proposals.