CodeLabeller: A Web-based Code Annotation Tool for Java Design Patterns and Summaries

The appropriate use of design patterns in code is a vital measurement of good software quality in object-oriented software applications. There exist tools to detect design pattern usage in Java source files, where their detection mechanisms have been honed through the use of supervised machine learning techniques that require large datasets of labelled files. However, manually labelling these files leads to issues such as tediousness if the team of labellers is small, and conflicting opinions between labellers, if large. Thus, we present CodeLabeller, a web-based tool which aims to provide a more efficient approach in handling the process of labelling Java source files at scale by improving the data collection process throughout, and improving the degree of reliability of responses by requiring each labeller to attach a confidence rating to each of their responses. We test CodeLabeller by constructing a corpus of over a thousand source files obtained from a large collection of open-source Java projects, and labelling each Java source file with their respective design patterns (if any), and summaries. This paper discusses the motivation behind thecreation of CodeLabeller, a demonstration of the tool and its UI, its implementation, benefits and lastly, some ideas for future improvements. A demo version of CodeLabeller can be found at: https://codelabeller.org.