Production Monitoring to Improve Test Suites

Software testing ensures that a software system behaves as intended. In this paper, we propose to use production executions to improve the quality of testing for certain methods of interest for developers. We devise an approach called PANKTI which monitors applications as they execute in production, and then automatically generates differential unit tests from the collected data. We implement our tool for Java and evaluate it on three real-world, open-source projects: a videoconferencing system, a PDF manipulation library, and an e-commerce application. We show that PANKTI is able to generate differential unit tests by monitoring target methods in production, and that the generated tests improve the quality of the test suite of the application under consideration.