We perform scalable approximate inference in continuous-depth Bayesian neural networks. In this model class, uncertainty about separate weights in each layer gives hidden units that follow a stochastic differential equation. We demonstrate gradient-based stochastic variational inference in this infinite-parameter setting, producing arbitrarily-flexible approximate posteriors. We also derive a novel gradient estimator that approaches zero variance as the approximate posterior over weights approaches the true posterior. This approach brings continuous-depth Bayesian neural nets to a competitive comparison against discrete-depth alternatives, while inheriting the memory-efficient training and tunable precision of Neural ODEs.
翻译:我们在连续深入的贝耶斯神经网络中进行可缩放的近似推论。 在这个模型类中,每个层的单重的不确定性给每个层提供了隐藏的单位,它们遵循的是随机差分方程。我们在这个无限参数设置中展示了基于梯度的随机多变推论,产生了任意灵活的近似后子体。我们还产生了一个新的梯度估计器,该测算器在接近真实的后背体时接近零差异。这个方法使连续深入的贝亚斯神经网与离散深度的替代物进行竞争性比较,同时继承了内核体的记忆高效培训和可测量精度。