Search-based test generation is guided by feedback from one or more fitness functions - scoring functions that judge solution optimality. Choosing informative fitness functions is crucial to meeting the goals of a tester. Unfortunately, many goals - such as forcing the class-under-test to throw exceptions, increasing test suite diversity, and attaining Strong Mutation Coverage - do not have effective fitness function formulations. We propose that meeting such goals requires treating fitness function identification as a secondary optimization step. An adaptive algorithm that can vary the selection of fitness functions could adjust its selection throughout the generation process to maximize goal attainment, based on the current population of test suites. To test this hypothesis, we have implemented two reinforcement learning algorithms in the EvoSuite unit test generation framework, and used these algorithms to dynamically set the fitness functions used during generation for the three goals identified above. We have evaluated our framework, EvoSuiteFIT, on a set of Java case examples. EvoSuiteFIT techniques attain significant improvements for two of the three goals, and show limited improvements on the third when the number of generations of evolution is fixed. Additionally, for two of the three goals, EvoSuiteFIT detects faults missed by the other techniques. The ability to adjust fitness functions allows strategic choices that efficiently produce more effective test suites, and examining these choices offers insight into how to attain our testing goals. We find that adaptive fitness function selection is a powerful technique to apply when an effective fitness function does not already exist for achieving a testing goal.
翻译:基于搜索的测试生成以一个或多个健身功能的反馈为指导,这些功能的评分可以判断最佳性。选择信息丰富的健身功能对于实现测试者的目标至关重要。不幸的是,许多目标,例如强迫受测试的班级推出例外,提高测试套件的多样性,以及实现强大的变异覆盖等,都没有有效的健身功能配方。我们提议,实现这些目标需要将健身功能识别作为二级优化步骤处理。一个适应性算法,可以改变健身功能的选择,从而根据当前测试套件的数量,在整个生成过程中调整其选择,以达到最大程度的目标。为了测试这一假设,我们在EvoSite单元生成框架中实施了两个强化学习算法,并利用这些算法来动态地设置为上述三个目标代代次设定的健身功能。我们评估了我们的框架,即EvoSiteFIT,以一组爪哇案例为例,将健身功能的识别作为二级优化步骤。EvoSiteFIT技术在三个目标中的两个目标中取得了显著的改进,在进化代数固定下来后,第三个阶段的改进幅度有限。此外,三个目标中的两个目标是,EvoStoiteF测试能够更高效地测试技术,从而实现其他的变更精度功能。