W-kernel and essential subspace for frequencist evaluation of Bayesian estimators

The posterior covariance matrix W defined by the log-likelihood of each observation plays important roles both in the sensitivity analysis and frequencist evaluation of the Bayesian estimators. This study is focused on the matrix W and its principal space; we term the latter as an essential subspace. A key tool for treating frequencist properties is the recently proposed Bayesian infinitesimal jackknife approximation (Giordano and Broderick (2023)). The matrix W can be interpreted as a reproducing kernel and is denoted as W-kernel. Using W-kernel, the essential subspace is expressed as a principal space given by the kernel principal component analysis. A relation to the Fisher kernel and neural tangent kernel is established, which elucidates the connection to the classical asymptotic theory. We also discuss a type of Bayesian-frequencist duality, which is naturally appeared from the kernel framework. Finally, two applications are discussed: the selection of a representative set of observations and dimensional reduction in the approximate bootstrap. In the former, incomplete Cholesky decomposition is introduced as an efficient method for computing the essential subspace. In the latter, different implementations of the approximate bootstrap for posterior means are compared.