It has been rightfully emphasized that the use of AI for clinical decision making could amplify health disparities. A machine learning model may pick up undesirable correlations, for example, between a patient's racial identity and clinical outcome. Such correlations are often present in (historical) data used for model development. There has been an increase in studies reporting biases in disease detection models across patient subgroups. Besides the scarcity of data from underserved populations, very little is known about how these biases are encoded and how one may reduce or even remove disparate performance. There is some speculation whether algorithms may recognize patient characteristics such as biological sex or racial identity, and then directly or indirectly use this information when making predictions. But it remains unclear how we can establish whether such information is actually used. This article aims to shed some light on these issues by exploring new methodology allowing intuitive inspections of the inner working of machine learning models for image-based detection of disease. We also evaluate an effective yet debatable technique for addressing disparities leveraging the automatic prediction of patient characteristics, resulting in models with comparable true and false positive rates across subgroups. Our findings may stimulate the discussion about safe and ethical use of AI.
翻译:人们正确地强调,将AI用于临床决策可能会扩大健康差异;机器学习模式可能会发现病人的种族特征和临床结果之间不可取的关联,例如,病人的种族认同和临床结果之间的关联。这种关联往往存在于用于模型开发的(历史)数据中。报告病人分组疾病检测模型偏差的研究有所增加。除了服务不足的人口缺乏数据之外,对于这些偏差是如何编码的以及如何减少或甚至消除分化性表现的知之甚少。有些猜测是,算法可能承认病人的特征,例如生物性别或种族特征,然后在作出预测时直接或间接地使用这种信息。但目前还不清楚我们如何能够确定这种信息是否实际使用。这一文章的目的是通过探索新方法,使对机器学习模型的内部工作进行直觉检查,以便进行基于图像的疾病检测。我们还评估了一种有效的、但不容置疑的办法来消除差异,用以利用对病人特征的自动预测,从而得出各分组之间真实和假正率的模型。我们的结论可能会促进关于安全使用AI的伦理性讨论。