Humans can reason compositionally when presented with new tasks. Previous research shows that appropriate prompting techniques enable large language models (LLMs) to solve artificial compositional generalization tasks such as SCAN. In this work, we identify additional challenges in more realistic semantic parsing tasks with larger vocabulary and refine these prompting techniques to address them. Our best method is based on least-to-most prompting: it decomposes the problem using prompting-based syntactic parsing, then uses this decomposition to select appropriate exemplars and to sequentially generate the semantic parse. This method allows us to set a new state of the art for CFQ while requiring only 1% of the training data used by traditional approaches. Due to the general nature of our approach, we expect similar efforts will lead to new results in other tasks and domains, especially for knowledge-intensive applications.
翻译:之前的研究显示, 适当的促动技术使大型语言模型( LLMs) 能够解决像 SCAN 这样的人为拼写概括化任务。 在这项工作中, 我们找出了在更现实的语义分析任务中更多的挑战, 使用更大的词汇, 并完善了这些催化技术来解决这些问题。 我们的最佳方法是基于最不切实际的方法: 它使用基于快速的合成法解析来分解问题, 然后使用这种分解来选择适当的示例和顺序生成语义分析。 这个方法使我们能够为 CFQ 设定新的艺术状态, 同时只需要传统方法所使用的培训数据的1% 。 由于我们的方法的一般性质, 我们期望类似的努力会在其它任务和领域带来新的结果, 特别是知识密集型的应用。