Emotions (e.g., Joy, Anger) are prevalent in daily software engineering (SE) activities, and are known to be significant indicators of work productivity (e.g., bug fixing efficiency). Recent studies have shown that directly applying general purpose emotion classification tools to SE corpora is not effective. Even within the SE domain, tool performance degrades significantly when trained on one communication channel and evaluated on another (e.g, StackOverflow vs. GitHub comments). Retraining a tool with channel-specific data takes significant effort since manually annotating large datasets of ground truth data is expensive. In this paper, we address this data scarcity problem by automatically creating new training data using a data augmentation technique. Based on an analysis of the types of errors made by popular SE-specific emotion recognition tools, we specifically target our data augmentation strategy in order to improve the performance of emotion recognition. Our results show an average improvement of 9.3% in micro F1-Score for three existing emotion classification tools (ESEM-E, EMTk, SEntiMoji) when trained with our best augmentation strategy.
翻译：情感（如喜悦、愤怒）在日常软件工程活动中普遍存在，且被证实是工作效率（例如缺陷修复效率）的重要指标。近期研究表明，将通用情感分类工具直接应用于软件工程语料库效果不佳。即使在软件工程领域内，当工具在一个通信渠道（如StackOverflow）上训练而在另一个渠道（如GitHub评论）上评估时，其性能也会显著下降。由于人工标注大规模真实数据成本高昂，使用渠道特定数据重新训练工具需要大量投入。本文通过采用数据增强技术自动生成新训练数据，以解决数据稀缺问题。基于对现有软件工程专用情感识别工具错误类型的分析，我们针对性地设计了数据增强策略，以提升情感识别性能。实验结果表明，采用我们最优增强策略训练后，三种现有情感分类工具（ESEM-E、EMTk、SEntiMoji）的微平均F1分数平均提升了9.3%。
相关内容
微信扫码咨询专知VIP会员