Natural language (NL) documentation is the bridge between software managers and testers, and NL test cases are prevalent in system-level testing and other quality assurance activities. Due to reasons such as requirements redundancy, parallel testing, and tester turnover within long evolving history, there are inevitably lots of redundant test cases, which significantly increase the cost. Previous redundancy detection approaches typically treat the textual descriptions as a whole to compare their similarity and suffer from low precision. Our observation reveals that a test case can have explicit test-oriented entities, such as tested function Components, Constraints, etc; and there are also specific relations between these entities. This inspires us with a potential opportunity for accurate redundancy detection. In this paper, we first define five test-oriented entity categories and four associated relation categories and re-formulate the NL test case redundancy detection problem as the comparison of detailed testing content guided by the test-oriented entities and relations. Following that, we propose Tscope, a fine-grained approach for redundant NL test case detection by dissecting test cases into atomic test tuple(s) with the entities restricted by associated relations. To serve as the test case dissection, Tscope designs a context-aware model for the automatic entity and relation extraction. Evaluation on 3,467 test cases from ten projects shows Tscope could achieve 91.8% precision, 74.8% recall, and 82.4% F1, significantly outperforming state-of-the-art approaches and commonly-used classifiers. This new formulation of the NL test case redundant detection problem can motivate the follow-up studies to further improve this task and other related tasks involving NL descriptions.
翻译:自然语言文件(NL)是软件管理员和测试者之间的桥梁,而NL测试案例在系统一级测试和其他质量保证活动中很普遍。由于需求冗余、平行测试和测试者在长期演变的历史中更替等原因,不可避免地有大量冗余测试案例,这些案例大大增加了成本。以前的冗余检测方法通常将文字描述作为一个整体,以比较其相似性和低精确度。我们的观察显示,测试案例可以有明确的测试导向实体,如测试的功能组成部分、制约等;这些实体之间也有着具体的关系。这激励我们有可能发现准确的冗余探测。在本文件中,我们首先界定了五个面向测试的实体类别和四个相关的关联关系类别,并重新配置NL测试测试案例的冗余检测问题,作为测试内容的详细比较,然后我们提出“范围”:通过将测试的测试案例分解成原子测试的测试案例,可以进一步改进与受相关关系限制的实体的原子测试后继任务。为了作为测试案例的分解剖,将NL测试样本的常规测试类型 4,Teroproal 10个测试案例的缩缩缩缩 。