The article is about algorithms for learning Bayesian hierarchical models, the computational complexity of which scales linearly with the number of observations and the number of parameters in the model. It focuses on crossed random effect and nested multilevel models, which are used ubiquitously in applied sciences, and illustrates the methodology on two challenging real data analyses on predicting electoral results and real estate prices respectively. The posterior dependence in both classes is sparse: in crossed random effects models it resembles a random graph, whereas in nested multilevel models it is tree-structured. For each class we develop a framework for scalable computation. We provide a number of negative (for crossed) and positive (for nested) results for the scalability (or lack thereof) of methods based on sparse linear algebra, which are relevant also to Laplace approximation methods for such models. Our numerical experiments compare with off-the-shelf variational approximations and Hamiltonian Monte Carlo. Our theoretical results, although partial, are useful in suggesting interesting methodologies and lead to conclusions that our numerics suggest to hold well beyond the scope of the underlying assumptions.
翻译:文章是关于学习巴伊西亚等级模型的算法, 其计算复杂性是用观测数量和模型参数数的线性尺度来计算。 它侧重于跨随机效应和嵌套多层次模型, 应用科学中普遍使用, 并展示了两种具有挑战性的实际数据分析方法, 分别用于预测选举结果和房地产价格。 两类的后继依赖性都很少: 在跨随机效应模型中, 它类似于随机图, 在嵌套的多层次模型中, 它是树形结构的。 对于每类, 我们开发了一个可缩放的计算框架。 我们提供了一些基于稀疏线性代数方法的可缩放性( 跨) 和正( 嵌套) 的结果, 这与这些模型的Laplace近似方法有关。 我们的数值实验与现成的变近似和汉密尔顿· 蒙特卡洛的数值比较, 虽然只是局部的, 但有助于提出有趣的方法, 并得出这样的结论, 我们的数字表明我们的数字显示远远超出基本假设的范围。