This paper revisits a fundamental problem in statistical inference from a non-asymptotic theoretical viewpoint $\unicode{x2013}$ the construction of confidence sets. We establish a finite-sample bound for the estimator, characterizing its asymptotic behavior in a non-asymptotic fashion. An important feature of our bound is that its dimension dependency is captured by the effective dimension $\unicode{x2013}$ the trace of the limiting sandwich covariance $\unicode{x2013}$ which can be much smaller than the parameter dimension in some regimes. We then illustrate how the bound can be used to obtain a confidence set whose shape is adapted to the optimization landscape induced by the loss function. Unlike previous works that rely heavily on the strong convexity of the loss function, we only assume the Hessian is lower bounded at optimum and allow it to gradually becomes degenerate. This property is formalized by the notion of generalized self-concordance which originated from convex optimization. Moreover, we demonstrate how the effective dimension can be estimated from data and characterize its estimation accuracy. We apply our results to maximum likelihood estimation with generalized linear models, score matching with exponential families, and hypothesis testing with Rao's score test.
翻译:本文重新审视了从不设防的理论观点 $\ unicode{ x2013}$ 构建信任制的统计推断中的一个根本问题。 我们为估算器设置了一个限定的缩略图, 以不设防的方式描述其非设防行为。 我们的缩略图的一个重要特征是, 其尺寸依赖性被有效的维度 $\ uncode{x2013}$( 美元) 所捕捉, 限定的三明治自相符合的痕迹 $\ unicode{x2013}$( 美元) 可能比某些制度的参数维度小得多 。 我们然后说明如何使用约束来获得一套符合损失功能所引致优化景观的一套信任。 与以往大量依赖损失功能的强烈共和的工程不同, 我们仅仅假设海珊的尺寸被较低的最佳尺寸约束, 并允许它逐渐退化。 这种属性由广义的自我调和自我调和概念的概念正式化。 此外, 我们证明有效的尺寸是如何从数据中估算并描述其估计精确度的准确性。 我们用最接近的指数的测算结果, 我们用最接近于指数的测算结果, 与最接近性的测算。