Consider a parametric model of distributions and the closest distribution in the model to the true distribution that is located outside the model. If we measure the closeness between two distributions with Kullback-Leibler divergence, the closest distribution is called "information projection" ([10]). The estimation risk of MLE is defined as the expectation of Kullback-Leibler divergence between the information projection and the predictive distribution with plugged-in MLE . We derived the asymptotic expansion of the risk up to the $n^{-2}$-order. On the other hand, we studied how small the divergence between the true distribution and the predictive distribution must be in order that Bayes error rate between the two distributions is guaranteed to be lower than a specified value. Combining these results, we proposed a criteria ("$p-n$ criteria") on whether MLE is sufficiently close to the information projection or not under the given model and the sample. Especially the criteria for an exponential family model is relatively simple and could be used for a complicated model without an explicit form of the normalizing constant. This criteria can be used as the solution to the sample size problem or the model acceptance (we also studied the relation of our results to the information criteria). We illustrated how to use the criteria through two practical data sets.
翻译：考虑模型中分布分布的参数模型以及模型中最接近模型外真实分布的分布模式。 如果我们测量两种分布在 Kullback- Leibler 差异之间的距离, 则最接近的分布被称为“ 信息预测 ” ( [10] ) 。 MLE 的估计风险被定义为 Kullback- Leiber 信息预测与插插入 MLE 的预测分布之间的预期差异。 我们从中推算出, 风险的无症状扩展到 $n ⁇ -2} $- order 。 另一方面, 我们研究了真实分布和预测分布之间的差差小, 以确保两种分布之间的贝斯错误率低于特定值。 将这些结果结合起来, 我们提出了一个标准( $- p- n$ 标准 ), 即 MLE 是否足够接近信息预测, 或没有在给定模型和样本下。 特别是指数式家庭模型的标准比较简单, 可以用于复杂的模型, 而没有明确的常态形式。 这个标准可以用来作为我们所研究的样本标准 的解决方案 。
相关内容
微信扫码咨询专知VIP会员