Statistical language models on source code have successfully assisted software engineering tasks. However, developers can create or pick arbitrary identifiers when writing source code. Freely chosen identifiers lead to the notorious out-of-vocabulary (OOV) problem that negatively affects model performance. Recently, Karampatsis et al. showed that using the Byte Pair Encoding (BPE) algorithm to address the OOV problem can improve the language models' predictive performance on source code. However, a drawback of BPE is that it cannot split the identifiers in a way that preserves the meaningful semantics. Prior researchers also show that splitting compound identifiers into sub-words that reflect the semantics can benefit software development tools. These two facts motivate us to explore whether identifier splitting techniques can be utilized to augment the BPE algorithm and boost the performance of open-vocabulary language models considered in Karampatsis et al.'s work. This paper proposes to split identifiers in both constructing vocabulary and processing model inputs procedures, thus exploiting three different settings of applying identifier splitting to language models for the code completion task. We contrast models' performance under these settings and find that simply inserting identifier splitting into the pipeline hurts the model performance, while a hybrid strategy combining identifier splitting and the BPE algorithm can outperform the original open-vocabulary models on predicting identifiers by 3.68% of recall and 6.32% of Mean Reciprocal Rank. The results also show that the hybrid strategy can improve the entropy of language models by 2.02%.
翻译:源代码的统计语言模型成功地帮助了软件工程任务。 但是, 开发者可以在写入源代码时创建或选择任意的标识符。 自由选择的标识符会导致臭名昭著的校外( OOOV)问题,对模型性能产生不利影响。 最近Karamptsis 等人指出, 使用 Byte Pair Encoding (BBE) 算法解决源代码的OOOV问题可以改进语言模型的预测性能。 但是, BPE 的一个缺点是, 它不能以保存有意义的语义代码的方式分割标识符。 以前的研究人员还表明, 将复合标识符分为反映语义的子字组(OOOOVV)问题,会有利于软件开发工具。 这两个事实激励我们探索, 是否可以利用识别符号分解技术来增强BPE的算法, 提高在Karamppatis & al. 工作中考虑的开放语言模型的预测性能。 本文建议, 在构建词汇和处理模型输入程序时, 利用三种不同的设置, 将识别符号分解到语言模型中的语言模型 6 代码完成任务 。 我们对比了BILIFF 战略, 将模型的绩效战略在这些模式下,, 将分析模型的性能, 通过这些模式的性能变化模式的性能变化模型的性能变化的性能 。