To understand how deep neural networks perform classification predictions, recent research attention has been focusing on developing techniques to offer desirable explanations. However, most existing methods cannot be easily applied for semantic segmentation; moreover, they are not designed to offer interpretability under the multi-annotator setting. Instead of viewing ground-truth pixel-level labels annotated by a single annotator with consistent labeling tendency, we aim at providing interpretable semantic segmentation and answer two critical yet practical questions: "who" contributes to the resulting segmentation, and "why" such an assignment is determined. In this paper, we present a learning framework of Tendency-and-Assignment Explainer (TAX), designed to offer interpretability at the annotator and assignment levels. More specifically, we learn convolution kernel subsets for modeling labeling tendencies of each type of annotation, while a prototype bank is jointly observed to offer visual guidance for learning the above kernels. For evaluation, we consider both synthetic and real-world datasets with multi-annotators. We show that our TAX can be applied to state-of-the-art network architectures with comparable performances, while segmentation interpretability at both levels can be offered accordingly.
翻译:为了了解深心神经网络如何进行分类预测,最近的研究注意力一直集中在开发技术以提供理想的解释。然而,大多数现有方法不能很容易地应用于语义分解;此外,这些方法在设计上不是为了在多批注的设置下提供可解释性。更具体地说,我们不是用一个标签趋势一致的单一说明者加注的地真像像像等级标签,而是用一个标记趋势一致的标记标志模型来了解地真真象等级标签,我们的目的是提供可解释的语义分解,回答两个关键但实际的问题:“谁”有助于由此产生的分解,以及“为什么”这种任务已经确定。在本文中,我们提出了一个用于在多批注和分配层次上提供可解释性解释性解释性说明的学习框架。更具体地说,我们学到了用于制作每种类型注注解趋势模型的进动内核子子,同时观察到一个原型银行为学习上面的内核。在评估中,我们考虑与多批注员一起使用合成和真实世界数据集。我们展示了用于可比较性结构的可同时,我们展示了TAX部分的可比较性结构。