The first step towards investigating the effectiveness of a treatment via a randomized trial is to split the population into control and treatment groups then compare the average response of the treatment group receiving the treatment to the control group receiving the placebo. In order to ensure that the difference between the two groups is caused only by the treatment, it is crucial that the control and the treatment groups have similar statistics. Indeed, the validity and reliability of a trial are determined by the similarity of two groups' statistics. Covariate balancing methods increase the similarity between the distributions of the two groups' covariates. However, often in practice, there are not enough samples to accurately estimate the groups' covariate distributions. In this paper, we empirically show that covariate balancing with the Standardized Means Difference (SMD) covariate balancing measure, as well as Pocock's sequential treatment assignment method, are susceptible to worst-case treatment assignments. Worst-case treatment assignments are those admitted by the covariate balance measure, but result in highest possible ATE estimation errors. We developed an adversarial attack to find adversarial treatment assignment for any given trial. Then, we provide an index to measure how close the given trial is to the worst-case. To this end, we provide an optimization-based algorithm, namely Adversarial Treatment ASsignment in TREatment Effect Trials (ATASTREET), to find the adversarial treatment assignments.
翻译:通过随机试验调查治疗有效性的第一步是将人口分为控制和治疗组,然后将接受治疗的治疗组的平均反应与接受安慰剂的控制组作比较。为了确保这两个组之间的差别只由治疗造成,控制和治疗组之间必须有类似的统计数字。事实上,试验的有效性和可靠性是由两个组的相似性统计数字决定的。共同平衡方法增加了两个组的共变差分布的相似性。然而,在实践中,往往没有足够的样本来准确估计接受治疗组的共变分布。在本文中,我们从经验上表明,与标准方法差异(SMD)的共变差平衡措施以及Pocock的相继治疗分配方法的平衡是容易发生最坏情况性治疗任务。最坏的治疗任务是由共变差平衡措施所接受的,但可能造成最高程度的ATE估计误差。我们开发了一种对抗性攻击,以找到任何给定的对口治疗组的分布。在本文中,我们从实验中提供了一种最接近的AVLA的指数,即最差的AVLA值。