Though end-to-end neural approaches have recently been dominating NLP tasks in both performance and ease-of-use, they lack interpretability and robustness. We propose Binder, a training-free neural-symbolic framework that maps the task input to a program, which (1) allows binding a unified API of language model (LM) functionalities to a programming language (e.g., SQL, Python) to extend its grammar coverage and thus tackle more diverse questions, (2) adopts an LM as both the program parser and the underlying model called by the API during execution, and (3) requires only a few in-context exemplar annotations. Specifically, we employ GPT-3 Codex as the LM. In the parsing stage, with only a few in-context exemplars, Codex is able to identify the part of the task input that cannot be answerable by the original programming language, correctly generate API calls to prompt Codex to solve the unanswerable part, and identify where to place the API calls while being compatible with the original grammar. In the execution stage, Codex can perform versatile functionalities (e.g., commonsense QA, information extraction) given proper prompts in the API calls. Binder achieves state-of-the-art results on WikiTableQuestions and TabFact datasets, with explicit output programs that benefit human debugging. Note that previous best systems are all finetuned on tens of thousands of task-specific samples, while Binder only uses dozens of annotations as in-context exemplars without any training. Our code is available at https://github.com/HKUNLP/Binder .
翻译:虽然端到端神经方法最近一直主导着NLP在性能和易用方面的任务,但它们缺乏解释性和稳健性。 我们提议Binder, 即一个将任务输入映射到程序上的培训性无神经- 同步框架, 它(1) 允许将统一的语言模型(LM)功能与编程语言(如SQL、Python)捆绑在一起, 以扩大其语法覆盖范围, 从而解决更为多样化的问题; (2) 采用一个LM, 既作为程序运行期间的程序解析器, 也作为AIP 所呼吁的基本模型; (3) 只需要几个在调解说性神经- 神经- 神经- 神经- 神经- 神经- 神经- 调制解调器, 使用GPT-3 代码作为LM 。 在解析阶段中, 代码可以识别任务输入部分无法被原始编程语言(如SQ. ), 任何调解算/ 调调调调 解析器的解算系统无法解析, 将A- drediplex- disalal- dismagial ladeal- lax lax lax lax lax lax lax lax lax lax lax lax lax lax lax lab lab lax lax lax lax lax lax lax lax lax lax lax lax labis