Mutation testing is an established software quality assurance technique for the assessment of test suites. While it is well-suited to estimate the general fault-revealing capability of a test suite, it is not practical and informative when the software under test must be validated against specific requirements. This is often the case for embedded software, where the software is typically validated against rigorously-specified safety properties. In such a scenario (i) a mutant is relevant only if it can impact the satisfaction of the tested properties, and (ii) a mutant is meaningfully-killed with respect to a property only if it causes the violation of that property. To address these limitations of mutation testing, we introduce property-based mutation testing, a method for assessing the capability of a test suite to exercise the software with respect to a given property. We evaluate our property-based mutation testing framework on Simulink models of safety-critical Cyber-Physical Systems (CPS) from the automotive and avionic domains and demonstrate how property-based mutation testing is more informative than regular mutation testing. These results open new perspectives in both mutation testing and test case generation of CPS.
翻译:变异测试是用来评估测试套件的既定软件质量保证技术; 虽然很适合估计测试套件的一般过错反应能力,但当测试中的软件必须对照具体要求进行验证时,它并不实用,信息也不丰富; 嵌入软件往往属于这种情况,软件通常根据严格规定的安全性能进行验证; 在这种假设中, (一) 变异体只有在能够影响测试的特性的满意度时才具有相关性; 和 (二) 变异体只有在导致侵犯该属性时,才对财产进行有意义的变异试验; 为解决突变测试的这些局限性,我们采用基于财产的变异测试,这是评估测试套件对特定财产应用软件的能力的一种方法; 我们评估我们从汽车和航空空间对安全临界网络-物理系统模型进行基于财产的变异测试框架,并证明基于财产的变异试验比定期变异试验更具有意义。