Safety Assessment for Autonomous System Perception Capabilities

Autonomous Systems (AS) are being increasingly proposed, or used, in Safety Critical (SC) applications, e.g., road vehicles. Many such systems make use of sophisticated sensor suites and processing to provide scene understanding which informs the AS' decision-making, e.g., path planning. The sensor processing typically makes use of Machine Learning (ML) and has to work in challenging environments, further the ML algorithms have known limitations, e.g., the possibility of false negatives or false positives in object classification. The well-established safety analysis methods developed for conventional SC systems are not well-matched to AS, ML, or the sensing systems used by AS. This paper proposes an adaptation of well-established safety analysis methods to address the specifics of sensing systems for AS, including addressing environmental effects and the potential failure modes of ML, and provides a rationale for choosing particular sets of guide words, or prompts, for safety analysis. It goes on to show how the results of the analysis can be used to inform the design and verification of the AS system and illustrates the new method by presenting a partial analysis of a mobile robot. The illustrations in the paper are primarily based on optical sensing, however the paper discusses the applicability of the method to other sensing modalities and its role in a wider safety process addressing the overall capabilities of AS