This paper presents an argument for why we are not measuring trust sufficiently in explainability, interpretability, and transparency research. Most studies ask participants to complete a trust scale to rate their trust of a model that has been explained/interpreted. If the trust is increased, we consider this a positive. However, there are two issues with this. First, we usually have no way of knowing whether participants should trust the model. Trust should surely decrease if a model is of poor quality. Second, these scales measure perceived trust rather than demonstrated trust. This paper showcases three methods that do a good job at measuring perceived and demonstrated trust. It is intended to be starting point for discussion on this topic, rather than to be the final say. The author invites critique and discussion.
翻译:本文说明了为什么我们没有充分衡量对解释性、可解释性和透明度研究的信任,大多数研究报告都要求参与者完成信任等级表,以评定他们对一个已经解释/解释的模型的信任程度。如果信任增加,我们认为这是一个积极的问题。然而,有两个问题。首先,我们通常无法知道参与者是否应该信任模型。如果模型质量差,信任肯定应该减少。第二,这些尺度衡量认为信任而不是证明信任。本文展示了三种方法,在衡量感知和显示信任方面做了很好的工作。它旨在成为讨论这个专题的起点,而不是最后一句话。作者请进行评论和讨论。