Metamorphic testing alleviates the test oracle problem by defining oracles based on relations (i.e., metamorphic relations) that govern multiple related inputs and their expected outputs. However, designing MRs requires domain-specific knowledge, which may not be generally available. This hinders the widespread adoption of MT. Fortunately, we observe that developers often embed such domain knowledge that encodes MRs in test cases, even without the awareness of metamorphic testing. These encoded MRs might be generalized and useful for testing the original programs (where these MRs are found) and other programs with similar functionalities. In this paper, we propose MR-Extractor, to automatically discover metamorphic relations from existing test cases in open-source software (OSS) projects. MR-Extractor firstly identifies MR-encoded test cases in OSS, and then generalizes encoded MRs and codifies them into parameterized methods (called codified MR) for MT. Finally, MR-Extractor validates the reusability of codified MRs for new test generation. Experimental results show that (i) over 12,000 MR-encoded test cases are identified by MR-Extractor, accounting for 1.26% of the total test cases, and (ii) test cases constructed from the MRs discovered by MR-Extractor can effectively improve test coverage. Compared with developer-written test suites, codified-MR-based test suites leads to a 25.97% and 25.83% increase in the line coverage and mutation score. When compared with Evosuite-generated test suites, codified-MR-based test suites even achieves an 83.11% increase in mutant score.
翻译:变形测试通过定义控制多个相关输入及其预期输出的关系(即变形关系),缓解了测试准则问题。但是,设计变形关系需要领域特定知识,这些知识可能并不普遍存在。这影响了变形测试的广泛采用。幸运的是,我们观察到开发人员通常会在测试用例中嵌入这种编码的领域知识,而不需要了解变形测试。这些编码的变形关系可以被概括,并且对于测试原始程序(其中包含这些变形关系)和具有类似功能的其他程序非常有用。在本文中,我们提出了MR-Extractor,以从开源软件(OSS)项目中的现有测试用例中自动发现变形关系。 MR-Extractor首先识别OSS中的MR编码的测试用例,然后将编码的MR概括并将其编码为MT的参数化方法(称为编码MR)。最后,MR-Extractor验证了编码MR的可重用性,以进行新测试生成。实验结果表明,(i)MR-Extractor识别了超过12,000个MR编码的测试用例,占总测试用例的1.26%,(ii)从MR-Extractor发现的MR构造的测试用例可以有效提高测试覆盖率。与开发人员编写的测试套件相比,基于编码-MR的测试套件引导线路覆盖率和易变得分分别增加了25.97%和25.83%。与Evosuite生成的测试套件相比,基于编码-MR的测试套件甚至取得了83.11%的突变得分增加。