We show that object detectors can hallucinate and detect missing objects; potentially even accurately localized at their expected, but non-existing, position. This is particularly problematic for applications that rely on visual part verification: detecting if an object part is present or absent. We show how popular object detectors hallucinate objects in a visual part verification task and introduce the first visual part verification dataset: DelftBikes, which has 10,000 bike photographs, with 22 densely annotated parts per image, where some parts may be missing. We explicitly annotated an extra object state label for each part to reflect if a part is missing or intact. We propose to evaluate visual part verification by relying on recall and compare popular object detectors on DelftBikes.
翻译:我们显示,天体探测器可以产生幻觉并探测失踪物体;甚至有可能精确定位于预期的、但不存在的位置。这对于依赖视觉部分核查的应用来说尤其有问题:检测物体部件是否存在;我们显示在视觉部分核查任务中流行的物体探测器致幻物体是如何在视觉部分核查任务中出现,并引入第一个视觉部分核查数据集:DelftBikes,它有10,000个自行车照片,每个图像有22个密集的附加说明部件,有些部分可能丢失。我们明确为每个部分附加一个额外的物体状态标签,以反映某一部件丢失或完好无损。我们提议通过在 DelftBikes 上对流行物体探测器进行回溯和比较来评价视觉部分核查。