Recently, the automated translation of source code from one programming language to another by using automatic approaches inspired by Neural Machine Translation (NMT) methods for natural languages has come under study. However, such approaches suffer from the same problem as previous NMT approaches on natural languages, viz. the lack of an ability to estimate and evaluate the quality of the translations; and consequently ascribe some measure of interpretability to the model's choices. In this paper, we attempt to estimate the quality of source code translations built on top of the TransCoder model. We consider the code translation task as an analog of machine translation (MT) for natural languages, with some added caveats. We present our main motivation from a user study built around code translation; and present a technique that correlates the confidences generated by that model to lint errors in the translated code. We conclude with some observations on these correlations, and some ideas for future work.
翻译:最近,正在研究使用神经机器翻译(NMT)方法启发的自然语言自动翻译源代码从一种编程语言到另一种语言的源代码自动化翻译方法,但这类方法与以前自然语言的NMT方法存在同样的问题,即缺乏估计和评价翻译质量的能力,因此将某种可解释的度量纳入模型的选择。在本文中,我们试图估算建在TransCoder模型之上的源代码翻译的质量。我们认为代码翻译工作是天然语言机器翻译的模拟,并附有一些附加说明。我们介绍了围绕代码翻译进行的用户研究的主要动机;我们提出了一种将该模型产生的信任与翻译代码中的误差联系起来的方法。我们最后对这些关联性提出了一些意见,并对未来工作提出了一些想法。