We study the benign overfitting theory in the prediction of the conditional average treatment effect (CATE), with linear regression models. As the development of machine learning for causal inference, a wide range of large-scale models for causality are gaining attention. One problem is that suspicions have been raised that the large-scale models are prone to overfitting to observations with sample selection, hence the large models may not be suitable for causal prediction. In this study, to resolve the suspicious, we investigate on the validity of causal inference methods for overparameterized models, by applying the recent theory of benign overfitting (Bartlett et al., 2020). Specifically, we consider samples whose distribution switches depending on an assignment rule, and study the prediction of CATE with linear models whose dimension diverges to infinity. We focus on two methods: the T-learner, which based on a difference between separately constructed estimators with each treatment group, and the inverse probability weight (IPW)-learner, which solves another regression problem approximated by a propensity score. In both methods, the estimator consists of interpolators that fit the samples perfectly. As a result, we show that the T-learner fails to achieve the consistency except the random assignment, while the IPW-learner converges the risk to zero if the propensity score is known. This difference stems from that the T-learner is unable to preserve eigenspaces of the covariances, which is necessary for benign overfitting in the overparameterized setting. Our result provides new insights into the usage of causal inference methods in the overparameterizated setting, in particular, doubly robust estimators.
翻译:我们研究了在预测有条件平均治疗效果(CATE)时的良性超称理论,并采用了线性回归模型。随着机器因果推断理论的发展,大量因果关系模型的大规模模型正在引起人们的注意。一个问题是,人们怀疑大型模型容易与抽样选择的观测相匹配,因此大型模型可能不适合因果预测。在这项研究中,为了解决可疑问题,我们通过应用最近的良性超配理论(Bartlett等人,2020年),调查过分模型的因果推断方法的有效性。具体地说,我们考虑其分布开关取决于分配规则的样本,并研究CATE的预测,而线性模型的尺寸与无限不同。我们侧重于两种方法:T-learner,它基于与每个治疗组分别构建的估算器之间的差异,以及偏差的概率(IPW)-升度,它解决了另一个近似于性分数的回归问题。在这两种方法中,Tsitemalitality-deal-depliator 的计算方法中,我们无法精确的排序,而我们无法精确的排序则显示我们的排序为直径比。