We study the relative entropy between the empirical estimate of a discrete distribution and the true underlying distribution. If the minimum value of the probability mass function exceeds an $\alpha > 0$ (i.e. when the true underlying distribution is bounded sufficiently away from the boundary of the simplex), we prove an upper bound on the moment generating function of the centered relative entropy that matches (up to logarithmic factors in the alphabet size and $\alpha$) the optimal asymptotic rates, subsequently leading to a sharp concentration inequality for the centered relative entropy. As a corollary of this result we also obtain confidence intervals and moment bounds for the centered relative entropy that are sharp up to logarithmic factors in the alphabet size and $\alpha$.
翻译:我们研究离散分布和真实基本分布的经验估计之间的相对酶值。如果概率质量函数的最小值超过1 $\ alpha > 0美元(即当真正的基本分布线与简单x的边界距离足够远时),我们证明在核心相对酶的生成功能的瞬间(与字母大小的对数系数和$/alpha$的对数系数相匹配),即最佳的亚麻率,从而导致中枢相对酶的高度集中不平等。作为这一结果的必然结果,我们还获得了中间相对酶的置信间隔和时间界限,而中间的相对酶在字母大小和$/alpha$的对数系数上均达到正对数系数。