How to trust auto-generated code patches? A developer survey and empirical assessment of existing program repair tools

Automated program repair is an emerging technology that seeks to automatically rectify bugs and vulnerabilities using learning, search, and semantic analysis. Trust in automatically generated patches is an important matter to drive greater adoption of program repair. Towards this goal, we conduct a survey with more than 100 software practitioners to understand the artifacts and setups needed to enhance trust in automatically generated patches. Based on the feedback from the survey, we quantitatively evaluate existing test-suite based program repair tools. We find that existing test-suite based repair tools cannot produce high-quality patches within a top-10 ranking and an acceptable time period of 1 hour. The developer feedback from our qualitative study and the observations from our quantitative examination of existing repair tools point to actionable insights to drive program repair research. Specifically, we note that producing repairs within an acceptable time-bound is very much dependent on leveraging an abstract search space representation of a rich enough search space. At the same time, while additional developer inputs are valuable to generate or rank patches, developers seem not to desire a significant human-in-the-loop interaction.