Tracking the Evolution of Static Code Warnings: the State-of-the-Art and a Better Approach

Static bug detection tools help developers detect problems in the code, including bad programming practices and potential defects. However, it is known that static bug detectors remain underutilized due to various reasons. Recent advances to incorporate static bug detectors in modern software development workflows, such as in code review and continuous integration, are shown capable of better motivating developers to fix the reported warnings on the fly. Moreover, tracking the static code warnings will benefit many downstream software engineering tasks, such as learning the fix patterns for automated program repair and learning which warnings are of more interest, so they can be prioritized automatically. Hence, precisely tracking the warnings by static bug detectors is critical to improve the utilization of static bug detectors further. In this paper, we study the effectiveness of the state-of-the-art (SOA) solution in tracking the warnings by static bug detectors and propose a better solution based on our analysis of the insufficiencies of the SOA solution. In particular, we examined over 2000 commits in four large-scale open-source systems (i.e., JClouds, Kafka, Spring-boot, and Guava) and crafted a dataset of 3,452 static code warnings by two static bug detectors (i.e., Spotbugs and PMD). We manually uncover the ground-truth evolution status of the static warnings: persistent, resolved, or newly-introduced. Moreover, upon manual analysis, we identified the main reasons behind the insufficiencies of the SOA solution. Finally, we propose a better approach to improving the tracking of static warnings over software development history. Our evaluation shows that our proposed approach provides a significant improvement in terms of the precision of the tracking, i.e., from 66.9% to 90.0%.