The likelihood-informed subspace (LIS) method offers a viable route to reducing the dimensionality of high-dimensional probability distributions arising in Bayesian inference. LIS identifies an intrinsic low-dimensional linear subspace where the target distribution differs the most from some tractable reference distribution. Such a subspace can be identified using the leading eigenvectors of a Gram matrix of the gradient of the log-likelihood function. Then, the original high-dimensional target distribution is approximated through various forms of marginalization of the likelihood function, in which the approximated likelihood only has support on the intrinsic low-dimensional subspace. This approximation enables the design of inference algorithms that can scale sub-linearly with the apparent dimensionality of the problem. Intuitively, the accuracy of the approximation, and hence the performance of the inference algorithms, are influenced by three factors -- the dimension truncation error in identifying the subspace, Monte Carlo error in estimating the Gram matrices, and Monte Carlo error in constructing marginalizations. This work establishes a unified framework to analyze each of these three factors and their interplay. Under mild technical assumptions, we establish error bounds for a range of existing dimension reduction techniques based on the principle of LIS. Our error bounds also provide useful insights into the accuracy of these methods. In addition, we analyze the integration of LIS with sampling methods such as Markov Chain Monte Carlo and sequential Monte Carlo. We also demonstrate the applicability of our analysis on a linear inverse problem with Gaussian prior, which shows that all the estimates can be dimension-independent if the prior covariance is a trace-class operator. Finally, we demonstrate various aspects of our theoretical claims on two nonlinear inverse problems.
翻译:概率知情的子空间 (LIS) 方法提供了一个可行的途径, 降低贝叶斯推论中产生的高维概率分布的维度。 LIS 确定了一个内在的低维线性子空间, 目标分布与某些可移植引用分布最不同。 这样的一个子空间可以使用日志类函数梯度的 Gram 矩阵的领先精度。 然后, 原始的高维目标分布会通过各种可能性功能的边缘化形式进行近似, 其中, 近似可能性仅在内在的低维度子空间上得到支持 。 这种近似可以设计下维线性线性亚空间, 使目标分布与某些可移植参考分布与某些可移植参考分布最相异 。 这种子空间的精确性可以由三个因素影响: 确定子空间的维度误差错误, Monte Carlo 估算Gramm 矩阵的误差, 以及 构建边缘化的 Monte Carlo 的错误。 这项工作建立了一个统一的框架, 来分析这三种因素中的非因素及其相近线性算算法, 在我们之前的精确的精确度分析中, 我们的精确度分析中, 的精确的精确度分析方法 也显示了我们之前的数值 的精确 。