While modern deep neural networks are performant perception modules, performance (accuracy) alone is insufficient, particularly for safety-critical robotic applications such as self-driving vehicles. Robot autonomy stacks also require these otherwise blackbox models to produce reliable and calibrated measures of confidence on their predictions. Existing approaches estimate uncertainty from these neural network perception stacks by modifying network architectures, inference procedure, or loss functions. However, in general, these methods lack calibration, meaning that the predictive uncertainties do not faithfully represent the true underlying uncertainties (process noise). Our key insight is that calibration is only achieved by imposing constraints across multiple examples, such as those in a mini-batch; as opposed to existing approaches which only impose constraints per-sample, often leading to overconfident (thus miscalibrated) uncertainty estimates. By enforcing the distribution of outputs of a neural network to resemble a target distribution by minimizing an $f$-divergence, we obtain significantly better-calibrated models compared to prior approaches. Our approach, $f$-Cal, outperforms existing uncertainty calibration approaches on robot perception tasks such as object detection and monocular depth estimation over multiple real-world benchmarks.
翻译:虽然现代深心神经网络是具有性能的感知模块,但性能(准确性)本身是不够的,特别是对于自我驾驶车辆等安全关键机器人应用而言。机器人自主性堆叠还要求这些否则是黑盒模型来产生可靠和校准的预测信任度。现有方法通过修改网络结构、推论程序或损失功能来估计神经网络感知堆的不确定性。然而,一般来说,这些方法缺乏校准,这意味着预测性不确定性并不忠实地代表真正的基本不确定性(过程噪音)。我们的关键见解是,校准只能通过对多种例子(如微型批量中的机器人)施加限制来实现。相对于仅对每个样本施加限制、往往导致过度信任(偏差)不确定性估计的现有方法而言,机器人认知任务(如天体探测和单体深度等)的不确定性校准方法。通过强制分配神经网络的产出,以与目标分布相近,我们获得的模型比以往方法要好得多的校准。我们的方法是,美元-Cal,超越了现有对机器人感知力校准方法,例如天体深度的物体探测和单体深度估计。