We develop E-variables for testing whether two or more data streams come from the same source or not, and more generally, whether the difference between the sources is larger than some minimal effect size. These E-variables lead to exact, nonasymptotic tests that remain safe, i.e. keep their type-I error guarantees, under flexible sampling scenarios such as optional stopping and continuation. In special cases our E-variables also have an optimal 'growth' property under the alternative. While the construction is generic, we illustrate it through the special case of k x 2 contingency tables, where we also allow for the incorporation of different restrictions on a composite alternative. Comparison to p-value analysis in simulations and a real-world example show that E-variables, through their flexibility, often allow for early stopping of data collection, thereby retaining similar power as classical methods, while also retaining the option of extending or combining data afterwards.
翻译:我们开发了电子变量,用于测试两个或两个以上数据流是否来自同一来源,以及更一般地测试来源之间的差别是否大于某些最小影响大小。这些电子变量导致精确、非抽取性测试,这些测试仍然安全,即保持其类型I的误差保障,在可选停止和继续等灵活抽样假设下进行。在特殊情况下,我们的电子变量在替代情况下也有最佳的“增长”属性。虽然构建是通用的,但我们通过k x 2 应急表格这一特殊案例加以说明,我们允许对复合替代数据采用不同的限制。在模拟和现实世界实例中,与p-价值分析进行比较表明,E变量通过灵活性,往往允许早期停止数据收集,从而保留与传统方法相似的“增长”属性,同时保留事后扩展或合并数据的选择。