Explainable Artificial Intelligence (XAI) aims to improve the transparency of machine learning (ML) pipelines. We systematize the increasingly growing (but fragmented) microcosm of studies that develop and utilize XAI methods for defensive and offensive cybersecurity tasks. We identify 3 cybersecurity stakeholders, i.e., model users, designers, and adversaries, who utilize XAI for 4 distinct objectives within an ML pipeline, namely 1) XAI-enabled user assistance, 2) XAI-enabled model verification, 3) explanation verification & robustness, and 4) offensive use of explanations. Our analysis of the literature indicates that many of the XAI applications are designed with little understanding of how they might be integrated into analyst workflows -- user studies for explanation evaluation are conducted in only 14% of the cases. The security literature sometimes also fails to disentangle the role of the various stakeholders, e.g., by providing explanations to model users and designers while also exposing them to adversaries. Additionally, the role of model designers is particularly minimized in the security literature. To this end, we present an illustrative tutorial for model designers, demonstrating how XAI can help with model verification. We also discuss scenarios where interpretability by design may be a better alternative. The systematization and the tutorial enable us to challenge several assumptions, and present open problems that can help shape the future of XAI research within cybersecurity.
翻译:解释性人工智能(XAI)的目的是提高机器学习(ML)管道的透明度。我们把日益增长(但支离破碎)的研究系统系统化,以开发和使用XAI方法进行防御性和攻击性网络安全任务。我们确定3个网络安全利益攸关方,即模型用户、设计师和对手,他们利用XAI实现ML管道中的4个不同目标,即:(1) XAI支持的用户援助,(2) XAI支持的模型核查,(3)解释性核查和稳健性,(4)解释性解释性使用。我们对文献的分析表明,许多XAI应用程序的设计对如何将其纳入分析工作流程知之甚少 -- -- 用户对解释性评价的研究只对14%的案件进行了。安全文献有时也无法混淆各种利益攸关方的作用,例如向模型用户和设计师提供解释性,同时使他们暴露在安全文献中特别最小化的作用。我们为此向模型设计师提供说明性辅导,说明XAI如何帮助我们进行模型核查。我们还可以讨论如何更好地解释这种解释性假设。我们还可以在XAI系统内部提出一些解释性设想,这样可以使解释性成为一种更难度。</s>