Hypothesis testing results often rely on simple, yet important assumptions about the behaviour of the distribution of p-values under the null and the alternative. We examine tests for one dimensional parameters of interest that converge to a normal distribution, possibly in the presence of nuisance parameters, and characterize the distribution of the p-values using techniques from the higher order asymptotics literature. We show that commonly held beliefs regarding the distribution of p-values are misleading when the variance and location of the test statistic are not well-calibrated or when the higher order cumulants of the test statistic are not negligible. Corrected tests are proposed and are shown to perform better than their first order counterparts in certain settings.
翻译:假设测试结果往往依赖于简单但重要的假设,根据无效值和替代值对p值分布的行为进行判断。我们检查与正常分布趋同的兴趣参数的一维参数的测试,可能时在有骚扰性参数的情况下进行,并使用高排序的Amptytotic文献中的技术对p值分布进行定性。我们表明,如果测试统计数据的差异和位置没有很好地校正,或者当测试统计数据中较高顺序的累积值不可忽略时,通常持有的关于p-val值分布的信念会产生误导作用。提出纠正性测试,并证明在某些环境下比第一顺序的对应值表现更好。