We perform scalable approximate inference in a recently-proposed family of continuous-depth Bayesian neural networks. In this model class, uncertainty about separate weights in each layer produces dynamics that follow a stochastic differential equation (SDE). We demonstrate gradient-based stochastic variational inference in this infinite-parameter setting, producing arbitrarily-flexible approximate posteriors. We also derive a novel gradient estimator that approaches zero variance as the approximate posterior approaches the true posterior. This approach further inherits the memory-efficient training and tunable precision of neural ODEs.
翻译:我们在最近提议的一个连续深入的贝耶斯神经网络的大家庭中进行了可伸缩的近似推论。在这个模型类别中,每个层的单重的不确定性产生动态,遵循随机差分方程(SDE)。我们在这个无限参数环境中展示了基于梯度的悬浮变异推论,产生了任意灵活近似近距离的子孙。我们还产生了一个新的梯度测算器,该测算仪在接近真实的后脑后部时接近零差异。这个方法进一步继承了神经元的记忆高效培训和可捕捉的精确度。