The aim of this note is to state a couple of general results about the properties of the penalized maximum likelihood estimators (pMLE) and of the posterior distribution for parametric models in a non-asymptotic setup and for possibly large or even infinite parameter dimension. We consider a special class of stochastically linear smooth (SLS) models satisfying two major conditions: the stochastic component of the log-likelihood is linear in the model parameter, while the expected log-likelihood is a smooth function. The main results simplify a lot if the expected log-likelihood is concave. For the pMLE, we establish a number of finite sample bounds about its concentration and large deviations as well as the Fisher and Wilks expansion. The later results extend the classical asymptotic Fisher and Wilks Theorems about the MLE to the non-asymptotic setup with large parameter dimension which can depend on the sample size. For the posterior distribution, our main result states a Gaussian approximation of the posterior which can be viewed as a finite sample analog of the prominent Bernstein--von Mises Theorem. In all bounds, the remainder is given explicitly and can be evaluated in terms of the effective sample size and effective parameter dimension. The results are dimension and coordinate free. In spite of generality, all the presented bounds are nearly sharp and the classical asymptotic results can be obtained as simple corollaries. Interesting cases of logit regression and of estimation of a log-density with smooth or truncation priors are used to specify the results and to explain the main notions.
翻译:本说明的目的是说明一些关于受罚的最大测算机(pMLE)的属性以及非无防线设置中参数模型以及可能大甚至无限参数维度参数的参数分布的后端分布等一些一般结果。我们认为,一个特殊的线性线性平滑模型(SLS)类别符合两个主要条件:在模型参数中,日志相似性(SLS)的随机部分是线性,而预期的日志相似性是一个平稳的函数。如果预期的正对数相似性是共通的,则主要结果会简化很多。对于 PMLE,我们建立一些关于其浓度和大偏差以及Fisherors和Wilks扩展的有限样本界限。后来的结果将经典的线性线性线性光线性(SLS)和威尔克斯理论(Wilks Torems) 扩展为非默认的参数设置, 取决于样本大小。对于后方分布, 我们的主要结果显示的是, 直线性对正向后端的对正值的直径直线性直线性直线性直径直径直线, 和直径直径直径直径直径直径对正的校对正的校正的校正的校正的校正的校正对结果可以被理解为, 。