Emotions (e.g., Joy, Anger) are prevalent in daily software engineering (SE) activities, and are known to be significant indicators of work productivity (e.g., bug fixing efficiency). Recent studies have shown that directly applying general purpose emotion classification tools to SE corpora is not effective. Even within the SE domain, tool performance degrades significantly when trained on one communication channel and evaluated on another (e.g, StackOverflow vs. GitHub comments). Retraining a tool with channel-specific data takes significant effort since manually annotating large datasets of ground truth data is expensive. In this paper, we address this data scarcity problem by automatically creating new training data using a data augmentation technique. Based on an analysis of the types of errors made by popular SE-specific emotion recognition tools, we specifically target our data augmentation strategy in order to improve the performance of emotion recognition. Our results show an average improvement of 9.3% in micro F1-Score for three existing emotion classification tools (ESEM-E, EMTk, SEntiMoji) when trained with our best augmentation strategy.
翻译：情感(例如,Joy, Anger)在日常软件工程(SE)活动中很普遍,已知是工作生产率的重要指标(例如,错误修复效率)。最近的研究表明,直接将一般目的情感分类工具应用于SE Corbora并不有效。即使是在SE域内,工具性能如果在一个通信频道上接受培训并对另一个通信频道进行评估(例如,StackOverplow vs. GitHub评论),工具性能也会显著降低。对一个带有频道特定数据的工具进行再培训需要大量努力,因为人工说明大量地面事实数据是昂贵的。在本文中,我们通过使用数据增强技术自动创建新的培训数据来解决数据稀缺问题。根据对流行的SE特定情感识别工具的错误类型的分析,我们具体针对我们的数据增强战略,以提高情绪认知的性能。我们的结果显示,在培训三种现有情感分类工具(ESEM-E, EMTk, SENNTIMoji)时,现有情感分类工具(ESEM-E-E)中微F1-S-1-S-STRAC)的平均改进了9.3%。
相关内容
微信扫码咨询专知VIP会员