Production Monitoring to Improve Test Suites

Software testing ensures that a software system behaves as intended. In this paper, we identify the methods in a software system that need better testing, and propose to use production executions to improve test suites. We devise an approach called PANKTI which monitors applications as they execute in production, and then automatically generates unit tests from the collected production data. We implement our tool for Java and evaluate it on three real-world, open-source projects: a videoconferencing system, a PDF manipulation library, and an e-commerce application. We show that PANKTI is able to generate unit tests for weakly-tested methods by monitoring them in production, and that the generated tests indeed improve the quality of the test suite of the application under consideration.