Cyber-physical systems (CPSs) typically consist of a wide set of integrated, heterogeneous components; consequently, most of their critical failures relate to the interoperability of such components.Unfortunately, most CPS test automation techniques are preliminary and industry still heavily relies on manual testing. With potentially incomplete, manually-generated test suites, it is of paramount importance to assess their quality. Though mutation analysis has demonstrated to be an effective means to assess test suite quality in some specific contexts, we lack approaches for CPSs. Indeed, existing approaches do not target interoperability problems and cannot be executed in the presence of black-box or simulated components, a typical situation with CPSs. In this paper, we introduce data-driven mutation analysis, an approach that consists in assessing test suite quality by verifying if it detects interoperability faults simulated by mutating the data exchanged by software components. To this end, we describe a data-driven mutation analysis technique (DaMAT) that automatically alters the data exchanged through data buffers. Our technique is driven by fault models in tabular form where engineers specify how to mutate data items by selecting and configuring a set of mutation operators. We have evaluated DaMAT with CPSs in the space domain; specifically, the test suites for the software systems of a microsatellite and nanosatellites launched on orbit last year. Our results show that the approach effectively detects test suite shortcomings, is not affected by equivalent and redundant mutants, and entails acceptable costs.
翻译:网络物理系统(CPS)通常由一套广泛、综合、多样的组件组成;因此,大多数关键故障都与这些组件的互操作性有关。 不幸的是,大多数CPS测试自动化技术都是初步的,工业仍然严重依赖人工测试。由于可能存在不完整、人工生成的测试套件,因此评估其质量至关重要。虽然突变分析证明是评估某些特定情况下测试套件质量的有效手段,但我们缺乏CPS的方法。事实上,现有方法并不针对互操作性问题,无法在黑盒或模拟组件存在的情况下执行。CPS的典型情况是。在本文件中,我们采用了数据驱动的突变分析方法,其中包括通过核查是否检测软件组件交换的数据所模拟的互操作性缺陷来评估一套测试套件的质量。为此,我们描述了一种数据驱动的突变分析技术(DaMAT),通过数据缓冲器自动改变数据交换的方法。我们的技术受到错误模型的驱动,在表格中,工程师们通过选择和测试系统测试系统测试系统测试系统系统,而不是系统测试系统内部测试系统。我们用SMATA测试系统系统,用来评估了系统。