To effectively optimize and personalize treatments, it is necessary to investigate the heterogeneity of treatment effects. With the wide range of users being treated over many online controlled experiments, the typical approach of manually investigating each dimension of heterogeneity becomes overly cumbersome and prone to subjective human biases. We need an efficient way to search through thousands of experiments with hundreds of target covariates and hundreds of breakdown dimensions. In this paper, we propose a systematic paradigm for detecting, surfacing and characterizing heterogeneous treatment effects. First, we detect if treatment effect variation is present in an experiment, prior to specifying any breakdowns. Second, we surface the most relevant dimensions for heterogeneity. Finally, we characterize the heterogeneity beyond just the conditional average treatment effects (CATE) by studying the conditional distributions of the estimated individual treatment effects. We show the effectiveness of our methods using simulated data and empirical studies.
翻译:为了有效地优化治疗方法并使之实现个性化,有必要调查治疗效果的异质性。由于许多在线控制实验对广泛的用户进行了治疗,人工调查异质性各个层面的典型方法过于繁琐,容易引起人类主观偏见。我们需要一种有效的方法,通过数百个目标共变和和数百个分解层面的数千个实验进行搜索。在本文件中,我们提出了一个用于检测、表面化和定性异质治疗效应的系统模式。首先,我们检测在说明任何分解之前,实验中是否存在治疗效果的变异。第二,我们展示了异质性性性最相关的层面。最后,我们通过研究个人治疗效应的估计有条件分布,我们用模拟数据和经验研究来描述我们方法的有效性。