Evaluating Developer-written Unit Test Case Reduction for Java -- A Replication Study

Abstract: Failing test case reduction can promote efficient debugging because a developer may not need to observe components that are not relevant to inducing failure. Failing test case reduction can also improve the efficiency of fault localization. These considerations have prompted researchers to study the reduction process, the reduction output, and the removed entities. Christi et al. studied test reduction using a tool called ReduSharptor for C# tests. They considered the test to be an Abstract Syntax Tree (AST). Based on that, they studied the reduction outcome and removed entities in terms of Leaf nodes and Non-Leaf nodes of the AST. They claimed that (1) leaf nodes are removed in large numbers, and (2) the probability of removal is slightly higher than non-leaf nodes. We replicate their results using a different test case reduction tool, ReduJavator, for Java unit tests. We evaluate test reduction using 30 randomly chosen bugs from the Defects4J database and 30 mutants for 6 open-source projects. Our results confirm their first claim: leaf nodes are removed in large numbers. Our results are inconclusive regarding their second claim; we cannot confirm that the probability of removal is higher for non-leaf nodes.