Hazard and Operability Analysis (HAZOP) is a powerful safety analysis technique with a long history in industrial process control domain. With the increasing use of Machine Learning (ML) components in cyber physical systems--so called Learning-Enabled Systems (LESs), there is a recent trend of applying HAZOP-like analysis to LESs. While it shows a great potential to reserve the capability of doing sufficient and systematic safety analysis, there are new technical challenges raised by the novel characteristics of ML that require retrofit of the conventional HAZOP technique. In this regard, we present a new Hierarchical HAZOP-Like method for LESs (HILLS). To deal with the complexity of LESs, HILLS first does "divide and conquer" by stratifying the whole system into three levels, and then proceeds HAZOP on each level to identify (latent-)hazards, causes, security threats and mitigation (with new nodes and guide words). Finally, HILLS attempts at linking and propagating the causal relationship among those identified elements within and across the three levels via both qualitative and quantitative methods. We examine and illustrate the utility of HILLS by a case study on Autonomous Underwater Vehicles, with discussions on assumptions and extensions to real-world applications. HILLS, as a first HAZOP-like attempt on LESs that explicitly considers ML internal behaviours and its interactions with other components, not only uncovers the inherent difficulties of doing safety analysis for LESs, but also demonstrates a good potential to tackle them.
翻译:危险和操作能力分析(HAZOP)是一种强大的安全分析技术,在工业过程控制领域具有悠久的历史。随着计算机物理系统中越来越多地使用机械学习(ML)组件,因此被称为学习-增强系统(LES),最近出现了将HAZOP类分析应用于LES的趋势。虽然它表明极有可能保留进行足够和系统的安全分析的能力,但ML的新特点带来了新的技术挑战,需要改造传统的HAZOP技术。在这方面,我们提出了一种新的HAZOP类高等级的LES(HELLS)方法。为了处理LES的复杂性,HILLS首先将整个系统分为三个层次,然后将HAZOP类的分析分为每个层次,确定(相对的)危险、原因、安全威胁和缓解(带有新的节点和指导词)。最后,HLLLS试图通过质量和定量的相互作用,处理LES中的三个层次内和跨层次的确定要素之间的因果关系。为了应对LIS的复杂程度,HLLS首先从质量和数量上分析,我们通过对HLSLS的实用性分析来明确研究。