Neural network ensembles, such as Bayesian neural networks (BNNs), have shown success in the areas of uncertainty estimation and robustness. However, a crucial challenge prohibits their use in practice. BNNs require a large number of predictions to produce reliable results, leading to a significant increase in computational cost. To alleviate this issue, we propose spatial smoothing, a method that spatially ensembles neighboring feature map points of convolutional neural networks. By simply adding a few blur layers to the models, we empirically show that spatial smoothing improves accuracy, uncertainty estimation, and robustness of BNNs across a whole range of ensemble sizes. In particular, BNNs incorporating spatial smoothing achieve high predictive performance merely with a handful of ensembles. Moreover, this method also can be applied to canonical deterministic neural networks to improve the performances. A number of evidences suggest that the improvements can be attributed to the stabilized feature maps and the smoothing of the loss landscape. In addition, we provide a fundamental explanation for prior works - namely, global average pooling, pre-activation, and ReLU6 - by addressing them as special cases of spatial smoothing. These not only enhance accuracy, but also improve uncertainty estimation and robustness by making the loss landscape smoother in the same manner as spatial smoothing. The code is available at https://github.com/xxxnell/spatial-smoothing.
翻译:Bayesian神经网络(BNNs)等神经网络集合体在不确定性估计和稳健性领域表现出成功。然而,一个关键的挑战是禁止在实践中使用这些网络。 BNNs需要大量的预测才能产生可靠的结果,从而导致计算成本的大幅上升。为了缓解这一问题,我们建议采取空间平滑方法,这种方法在空间上将动态神经网络的相邻特征地图点聚集在一起。通过简单地为模型添加一些模糊的层次,我们从经验上表明空间平滑可以提高BNNS的准确性、不确定性估计和稳健性。特别是,包含空间平滑性的BNNS需要大量预测才能产生高的预测性能,而只是使用少量的星团组合。此外,这种方法还可以应用于Canonic 确定性神经网络,以改善性能。一些证据表明,改进的改进可以归因于稳定的特征地图和损失景观的平滑性。此外,我们还为先前的工程提供了基本的解释,即全球平均平滑性组合、平滑性平滑性估算法则通过特殊案例加以改进。