Large language models have demonstrated outstanding performance on a wide range of tasks such as question answering and code generation. On a high level, given an input, a language model can be used to automatically complete the sequence in a statistically-likely way. Based on this, users prompt these models with language instructions or examples, to implement a variety of downstream tasks. Advanced prompting methods can even imply interaction between the language model, a user, and external tools such as calculators. However, to obtain state-of-the-art performance or adapt language models for specific tasks, complex task- and model-specific programs have to be implemented, which may still require ad-hoc interaction. Based on this, we present the novel idea of Language Model Programming (LMP). LMP generalizes language model prompting from pure text prompts to an intuitive combination of text prompting and scripting. Additionally, LMP allows constraints to be specified over the language model output. This enables easy adaption to many tasks, while abstracting language model internals and providing high-level semantics. To enable LMP, we implement LMQL (short for Language Model Query Language), which leverages the constraints and control flow from an LMP prompt to generate an efficient inference procedure that minimizes the number of expensive calls to the underlying language model. We show that LMQL can capture a wide range of state-of-the-art prompting methods in an intuitive way, especially facilitating interactive flows that are challenging to implement with existing high-level APIs. Our evaluation shows that we retain or increase the accuracy on several downstream tasks, while also significantly reducing the required amount of computation or cost in the case of pay-to-use APIs (13-85% cost savings).
翻译:大型语言模型在诸如答题和代码生成等广泛任务中表现出了杰出的绩效。 在高层次上,如果输入,可以使用一种语言模型来自动以统计上可能的方式完成序列。 在此基础上, 用户会用语言指令或示例来推动这些模型, 以实施各种下游任务。 高级促进方法甚至可以意味着语言模型、 用户和计算器等外部工具之间的互动。 然而, 要获得最新业绩或为具体任务调整语言模型, 复杂的任务和模型程序必须实施, 这可能需要临时的精确互动。 基于此, 我们展示了语言模型程序的新概念( LMP ) 。 从纯文本提示或示例到文本提示和脚本的直观组合。 此外, LMP 允许对语言模型输出进行限制。 这样可以方便地适应许多任务,同时抽象语言模型内部和高层次的描述。 为了让 LMP 、 我们实施LMQL 流, 并且 快速的计算方法, 也能够大大降低成本水平, 在语言模型操作中, 快速的LMLL 程序可以显示一个快速的流程, 。