We perform scalable approximate inference in a continuous-depth Bayesian neural network family. In this model class, uncertainty about separate weights in each layer gives hidden units that follow a stochastic differential equation. We demonstrate gradient-based stochastic variational inference in this infinite-parameter setting, producing arbitrarily-flexible approximate posteriors. We also derive a novel gradient estimator that approaches zero variance as the approximate posterior over weights approaches the true posterior. This approach brings continuous-depth Bayesian neural nets to a competitive comparison against discrete-depth alternatives, while inheriting the memory-efficient training and tunable precision of Neural ODEs.
翻译:我们在一个连续深入的贝耶斯神经网络大家庭中进行可缩放的近似推论。 在这个模型类中,每个层的单重的不确定性给每个层提供了隐藏的单位,它们遵循的是随机差分方程。 我们在这个无限参数设置中展示了基于梯度的随机多变推论,产生了任意灵活的近似后子体。 我们还产生了一个新的梯度估计器,它接近零差异,因为近似后背体的重量过重接近真正的后背体。 这种方法使连续深入的贝亚神经网与离散深度的替代物进行竞争性比较,同时继承了记忆高效的培训和神经极值的金枪鱼精度。