Hitherto statistical type inference systems rely thoroughly on supervised learning approaches, which require laborious manual effort to collect and label large amounts of data. Most Turing-complete imperative languages share similar control- and data-flow structures, which make it possible to transfer knowledge learned from one language to another. In this paper, we propose a cross-lingual transfer learning framework, PLATO, for statistical type inference, which allows us to leverage prior knowledge learned from the labeled dataset of one language and transfer it to the others, e.g., Python to JavaScript, Java to JavaScript, etc. PLATO is powered by a novel kernelized attention mechanism to constrain the attention scope of the backbone Transformer model such that model is forced to base its prediction on commonly shared features among languages. In addition, we propose the syntax enhancement that augments the learning on the feature overlap among language domains. Furthermore, PLATO can also be used to improve the performance of the conventional supervised-based type inference by introducing cross-language augmentation, which enables the model to learn more general features across multiple languages. We evaluated PLATO under two settings: 1) under the cross-domain scenario that the target language data is not labeled or labeled partially, the results show that PLATO outperforms the state-of-the-art domain transfer techniques by a large margin, e.g., it improves the Python to TypeScript baseline by +14.6%@EM, +18.6%@weighted-F1, and 2) under the conventional monolingual supervised scenario, PLATO improves the Python baseline by +4.10%@EM, +1.90%@weighted-F1 with the introduction of the cross-lingual augmentation.
翻译:14 统计类型推断系统完全依赖于监管的学习方法,这需要人工努力收集和标签大量数据。大多数图灵完整的必备语言都拥有类似的控制和数据流结构,从而有可能将从一种语言学到的知识传输到另一种语言。在本文中,我们提议了一个跨语言传输学习框架PLATO,用于统计类型推断,使我们能够利用从一种语言的标签数据集中学到的先前知识,并将其传输到其他语言,例如,Python到 JavaScript, JavaScript 等。PLATO 是一个全新的控制关注机制,以限制骨干变换模式的注意范围,这种模式被迫将其预测建立在语言之间共同共有的特征之上。此外,我们提议加强语系税,增加语言区域间特征重叠的学习。此外,PLATO 还可以使用常规监督型的引入语言增强功能,使模型能够学习跨语言的更一般的基流流, PLATO 将O 的基流转换到跨域的基线,我们通过两个基域的基调的基调变的PLA,我们用PLA-O 数据来评估了O 。