Multimodal program synthesis, which leverages different types of user input to synthesize a desired program, is an attractive way to scale program synthesis to challenging settings; however, it requires integrating noisy signals from the user, like natural language, with hard constraints on the program's behavior. This paper proposes an optimal neural synthesis approach where the goal is to find a program that satisfies user-provided constraints while also maximizing the program's score with respect to a neural model. Specifically, we focus on multimodal synthesis tasks in which the user intent is expressed using a combination of natural language (NL) and input-output examples. At the core of our method is a top-down recurrent neural model that places distributions over abstract syntax trees conditioned on the NL input. This model not only allows for efficient search over the space of syntactically valid programs, but it allows us to leverage automated program analysis techniques for pruning the search space based on infeasibility of partial programs with respect to the user's constraints. The experimental results on a multimodal synthesis dataset (StructuredRegex) show that our method substantially outperforms prior state-of-the-art techniques in terms of accuracy and efficiency, and finds model-optimal programs more frequently.
翻译:多式程序合成利用不同种类的用户投入来合成理想的程序,是将程序合成规模扩大到具有挑战性的环境的一种有吸引力的方法;然而,它需要将用户的噪音信号(如自然语言)与对程序行为的严格限制结合起来。本文件建议了一种最佳神经合成方法,其目标是找到一个满足用户提供的制约因素的程序,同时最大限度地提高程序在神经模型方面的得分。具体地说,我们侧重于多式联运合成任务,即用户意图是使用自然语言(NL)和输入输出实例相结合来表达的。我们的方法核心是一个自上而下的经常性神经模型,将分布置于以 NL 输入为条件的抽象合成树之上。这一模型不仅允许在同步有效程序空间上进行有效搜索,而且还使我们能够利用自动程序分析技术,根据部分程序对用户制约的不可行性来调整搜索空间。在多式合成数据集(结构调整Regex)上进行的实验结果显示,我们的方法的精确度远远高于先前的精确度,在模型中发现我们的方法,并且更精确性地表现了先前的精确性程序。