In this paper, we explore adaptive inference based on variational Bayes. Although a number of studies have been conducted to analyze contraction properties of variational posteriors, there is still a lack of a general and computationally tractable variational Bayes method that can achieve adaptive optimal contraction of the variational posterior. We propose a novel variational Bayes framework, called adaptive variational Bayes, which can operate on a collection of models with varying dimensions and structures. The proposed framework combines variational posteriors over individual models with certain weights to obtain a variational posterior over the entire model. It turns out that this combined variational posterior minimizes the Kullback-Leibler divergence to the original posterior distribution. We show that the proposed variational posterior achieves optimal contraction rates adaptively under very general conditions and attains model selection consistency when the true model structure exists. We apply the general results obtained for the adaptive variational Bayes to several examples including deep learning models and derive some new and adaptive inference results. Moreover, we consider the use of quasi-likelihood in our framework. We formulate conditions on the quasi-likelihood to ensure the adaptive optimality and discuss specific applications to stochastic block models and nonparametric regression with sub-Gaussian errors.
翻译:在本文中,我们探讨了基于变异贝壳的适应性推断。虽然已经进行了一些研究来分析变异后继体的收缩特性,但是仍然缺乏一种一般的和可计算可移动的变异贝壳方法,能够实现变异后继体的适应性最佳收缩。我们提出了一个新的变异贝壳框架,称为适应性变异贝壳,该框架可以在一系列具有不同维度和结构的模型的基础上运作。拟议框架将变异后继体与个别模型的变异后继体结合起来,并具有一定的重量,以获得整个模型的变异后继体。我们发现,这种变异后继-利差的组合将最初的后继体分布差异减至最小。我们表明,拟议的变异后继体在非常一般的条件下实现最佳收缩率,并在真实模型结构存在时实现模型选择的一致性。我们将适应性变异系后继体的一般结果应用于几个例子,包括深层次的学习模型,并得出一些新的和适应性推论结果。此外,我们考虑在框架中如何使用准相似的后继力,我们用准后退变模型来讨论不至最优的模型。