Two-sample testing tests whether the distributions generating two samples are identical. We pose the two-sample testing problem in a new scenario where the sample measurements (or sample features) are inexpensive to access, but their group memberships (or labels) are costly. We devise the first \emph{active sequential two-sample testing framework} that not only sequentially but also \emph{actively queries} sample labels to address the problem. Our test statistic is a likelihood ratio where one likelihood is found by maximization over all class priors, and the other is given by a classification model. The classification model is adaptively updated and then used to guide an active query scheme called bimodal query to label sample features in the regions with high dependency between the feature variables and the label variables. The theoretical contributions in the paper include proof that our framework produces an \emph{anytime-valid} $p$-value; and, under reachable conditions and a mild assumption, the framework asymptotically generates a minimum normalized log-likelihood ratio statistic that a passive query scheme can only achieve when the feature variable and the label variable have the highest dependence. Lastly, we provide a \emph{query-switching (QS)} algorithm to decide when to switch from passive query to active query and adapt bimodal query to increase the testing power of our test. Extensive experiments justify our theoretical contributions and the effectiveness of QS.
翻译:以两个模样测试产生两个样本的分布是否完全相同。 我们在一个新的假设中提出两个模样测试问题, 在新的假设中, 样本测量( 或样本特征) 价格低廉, 但其组成员( 或标签) 费用高。 我们设计了第一个 emph{ 活性顺序顺序 双样样测试框架}, 不仅按顺序进行测试, 而且还按顺序进行 emph{ 活性询问} 样本标签, 以解决问题。 我们的测试统计是一个可能性比, 一种可能性是通过对所有类前题的最大化发现, 而另一种可能性则由分类模型提供。 分类模型是适应性更新的, 然后用来指导一个名为双调查询的动态查询方案, 在特性变量和标签变量变量变量变量之间高度依赖的区域, 本文的理论贡献包括证明我们的框架产生了一个 emph{ 时间- valid} $p- 价值 ; 在可实现的条件和温和假设下, 框架产生一个最小的标准化的逻辑比值比值比值比值比值比值比值比值比值比值比值比值比值比值比值比值比值比值。 当我们进行被动的试性查询方案在特性变数测试时, 我们的变数和变数调算算算算算算算算算算算算算法的系统只能的系统只能在的系统只能只能只能在最后才算算算算算算算算算算算算算算算算算算算得得得得得最高时, Q。