Estimating personalized effects of treatments is a complex, yet pervasive problem. To tackle it, recent developments in the machine learning (ML) literature on heterogeneous treatment effect estimation gave rise to many sophisticated, but opaque, tools: due to their flexibility, modularity and ability to learn constrained representations, neural networks in particular have become central to this literature. Unfortunately, the assets of such black boxes come at a cost: models typically involve countless nontrivial operations, making it difficult to understand what they have learned. Yet, understanding these models can be crucial -- in a medical context, for example, discovered knowledge on treatment effect heterogeneity could inform treatment prescription in clinical practice. In this work, we therefore use post-hoc feature importance methods to identify features that influence the model's predictions. This allows us to evaluate treatment effect estimators along a new and important dimension that has been overlooked in previous work: We construct a benchmarking environment to empirically investigate the ability of personalized treatment effect models to identify predictive covariates -- covariates that determine differential responses to treatment. Our benchmarking environment then enables us to provide new insight into the strengths and weaknesses of different types of treatment effects models as we modulate different challenges specific to treatment effect estimation -- e.g. the ratio of prognostic to predictive information, the possible nonlinearity of potential outcomes and the presence and type of confounding.
翻译:估计治疗的个性效应是一个复杂而普遍的问题。要解决这一问题,机器学习(ML)关于不同治疗效果估计的文献的最近发展产生了许多复杂但不透明的工具:由于其灵活性、模块性以及学习受限制的表达方式的能力,神经网络尤其成为了该文献的中心。不幸的是,这类黑盒的资产是成本高的:模型通常涉及无数次非细微操作,难以理解它们所学到的东西。然而,理解这些模型可能是至关重要的 -- -- 例如,在医学方面,所发现的关于治疗效果异质性的知识可以为临床实践的治疗处方提供参考。因此,我们在此工作中,使用后热特征重要性方法来查明影响模型预测的特征。这使我们能够评估在以前工作中被忽视的一个新和重要层面的治疗效应估计因素:我们建立一个基准环境,以经验调查个人化治疗效果模型确定预测变量的能力 -- -- 将确定对治疗的不同反应。然后,我们的基准环境使我们能够对不同类型治疗结果的强性和弱点进行新的了解,从而能够对不同类型治疗结果的弱点作出新的认识。