Bayesian neural networks have shown great promise in many applications where calibrated uncertainty estimates are crucial and can often also lead to a higher predictive performance. However, it remains challenging to choose a good prior distribution over their weights. While isotropic Gaussian priors are often chosen in practice due to their simplicity, they do not reflect our true prior beliefs well and can lead to suboptimal performance. Our new library, BNNpriors, enables state-of-the-art Markov Chain Monte Carlo inference on Bayesian neural networks with a wide range of predefined priors, including heavy-tailed ones, hierarchical ones, and mixture priors. Moreover, it follows a modular approach that eases the design and implementation of new custom priors. It has facilitated foundational discoveries on the nature of the cold posterior effect in Bayesian neural networks and will hopefully catalyze future research as well as practical applications in this area.
翻译:在许多应用中,校准的不确定性估计至关重要,而且往往还可能导致更高的预测性性能,但选择优于其重量的先前分布仍然具有挑战性。虽然在实际中通常选择异热带高斯前科,但由于这些前科的简单性,这些前科并不反映我们真正的先入之见,并可能导致不理想的性能。我们的新图书馆,即BNNPriors,使Bayesian神经网络上最先进的马可夫链蒙特卡洛导火力得以在有广泛预设的前科的Bayesian神经网络上进行最先进的马可夫链蒙特卡洛导火力,这些前科包括重尾科、等级和混合前科。此外,它采取模块化方法,方便了新习惯前科的设计和实施。它促进了关于Bayesian神经网络冷后部效应性质的基本发现,并有望催化未来研究以及该领域的实际应用。