The connection between Bayesian neural networks and Gaussian processes gained a lot of attention in the last few years, with the flagship result that hidden units converge to a Gaussian process limit when the layers width tends to infinity. Underpinning this result is the fact that hidden units become independent in the infinite-width limit. Our aim is to shed some light on hidden units dependence properties in practical finite-width Bayesian neural networks. In addition to theoretical results, we assess empirically the depth and width impacts on hidden units dependence properties.
翻译:Bayesian神经网络和Gaussian过程之间的联系在过去几年中引起了许多关注,其旗舰结果是,当层宽度趋向无限时,隐藏的单元会汇合到高斯过程的极限。这一结果的基础是隐藏的单元在无限宽限中变得独立。我们的目标是在实际的有限宽度Bayesian神经网络中说明隐藏的单位依赖特性。除了理论结果外,我们还从经验角度评估对隐藏单元依赖特性的深度和宽度影响。