Inductive reasoning is a core component of human intelligence. In the past research of inductive reasoning within computer science, logic language is used as representations of knowledge (facts and rules, more specifically). However, logic language can cause systematic problems for inductive reasoning such as disability of handling raw input such as natural language, sensitiveness to mislabeled data, and incapacity to handle ambiguous input. To this end, we propose a new task, which is to induce natural language rules from natural language facts, and create a dataset termed DEER containing 1.2k rule-fact pairs for the task, where rules and facts are written in natural language. New automatic metrics are also proposed and analysed for the evaluation of this task. With DEER, we investigate a modern approach for inductive reasoning where we use natural language as representation for knowledge instead of logic language and use pretrained language models as ''reasoners''. Moreover, we provide the first and comprehensive analysis of how well pretrained language models can induce natural language rules from natural language facts. We also propose a new framework drawing insights from philosophy literature for this task, which we show in the experiment section that surpasses baselines in both automatic and human evaluations.
翻译:感应推理是人类智力的核心组成部分。在过去对计算机科学中的感性推理的研究中,逻辑语言被用来作为知识的体现(更具体地说,逻辑语言),但逻辑语言可能会造成系统的推理问题,例如处理自然语言等原始输入的残疾,对错误标签数据敏感,以及没有能力处理模糊输入。为此,我们提出一项新的任务,即从自然语言事实中引出自然语言规则,并创建一个称为DEER的数据集,其中含有以自然语言书写规则和事实的1.2k规则-事实组合。还提出和分析新的自动衡量标准,用于评估这项任务。与DEER一起,我们调查现代推理方法,即我们使用自然语言作为知识代表,而不是逻辑语言,以及使用预先训练的语言模型作为“解释者”。此外,我们首次全面分析了预先训练的语言模型如何从自然语言事实中引出自然语言规则。我们还提出了一个新的框架,从哲学文献中得出关于这项任务的深刻见解,我们在试验部分显示,在自动和人类评价中超越基线。