Bayesian neural networks (BNNs) can account for both aleatoric and epistemic uncertainty. However, in BNNs the priors are often specified over the weights which rarely reflects true prior knowledge in large and complex neural network architectures. We present a simple approach to incorporate prior knowledge in BNNs based on external summary information about the predicted classification probabilities for a given dataset. The available summary information is incorporated as augmented data and modeled with a Dirichlet process, and we derive the corresponding \emph{Summary Evidence Lower BOund}. The approach is founded on Bayesian principles, and all hyperparameters have a proper probabilistic interpretation. We show how the method can inform the model about task difficulty and class imbalance. Extensive experiments show that, with negligible computational overhead, our method parallels and in many cases outperforms popular alternatives in accuracy, uncertainty calibration, and robustness against corruptions with both balanced and imbalanced data.
翻译:Bayesian 神经网络(BNNs) 既可以说明偏振性不确定性,也可以说明认知性不确定性。然而,在 BNNs 中,先行信息通常在权重上具体指明,这些权重很少反映大型和复杂的神经网络结构中真实的先前知识。我们提出了一个简单的方法,根据关于某一数据集预计分类概率的外部摘要信息,将BNs先前的知识纳入BNs。现有的摘要信息是作为增强的数据纳入的,并采用Drichlet 进程的模式,我们得出相应的 \emph{ Summary evidence down}。该方法以Bayesian 原则为基础,所有超光量参数都有适当的概率解释。我们展示了该方法如何为任务难度和分类不平衡的模式提供信息。广泛的实验表明,在可忽略的计算间接成本的情况下,我们的方法平行,在许多情况下在精确性、不确定性校准和稳健性方面超过了流行的替代方法,同时使用平衡和不平衡的数据。