The goal of many scientific experiments including A/B testing is to estimate the average treatment effect (ATE), which is defined as the difference between the expected outcomes of two or more treatments. In this paper, we consider a situation where an experimenter can assign a treatment to research subjects sequentially. In adaptive experimental design, the experimenter is allowed to change the probability of assigning a treatment using past observations for estimating the ATE efficiently. However, with this approach, it is difficult to apply a standard statistical method to construct an estimator because the observations are not independent and identically distributed. We thus propose an algorithm for efficient experiments with estimators constructed from dependent samples. We also introduce a sequential testing framework using the proposed estimator. To justify our proposed approach, we provide finite and infinite sample analyses. Finally, we experimentally show that the proposed algorithm exhibits preferable performance.
翻译:包括A/B测试在内的许多科学实验的目的,是估计平均治疗效果(ATE),它被界定为两种或两种以上治疗的预期结果之间的差别。在本文中,我们考虑了实验者可以按顺序对研究对象进行治疗的情况。在适应性实验设计中,允许实验者改变利用过去观察来有效估计ATE的治疗概率。然而,采用这种方法,很难采用标准统计方法来构建一个估计结果,因为观测不是独立和分布相同的。因此,我们建议一种算法,用从依赖性样品中建造的测算器进行高效的实验。我们还采用一个顺序测试框架,使用拟议的测算仪来说明我们拟议方法的合理性。我们提供有限和无限的抽样分析。最后,我们实验性地表明,提议的算法显示了较好的性能。