A longstanding question in cognitive science concerns the learning mechanisms underlying compositionality in human cognition. Humans can infer the structured relationships (e.g., grammatical rules) implicit in their sensory observations (e.g., auditory speech), and use this knowledge to guide the composition of simpler meanings into complex wholes. Recent progress in artificial neural networks has shown that when large models are trained on enough linguistic data, grammatical structure emerges in their representations. We extend this work to the domain of mathematical reasoning, where it is possible to formulate precise hypotheses about how meanings (e.g., the quantities corresponding to numerals) should be composed according to structured rules (e.g., order of operations). Our work shows that neural networks are not only able to infer something about the structured relationships implicit in their training data, but can also deploy this knowledge to guide the composition of individual meanings into composite wholes.
翻译:认知科学中长期存在的一个问题涉及人类认知构成的学习机制。 人类可以推断感官观察(如听觉讲话)中隐含的结构化关系(如语法规则),并用这种知识指导更简单的含义组成复杂的整体。 人工神经网络的最近进展表明,当大型模型经过足够的语言数据培训时,其表达方式会出现语法结构。 我们将这项工作扩大到数学推理领域,从而有可能就含义(如数字的相对数量)如何按照结构化规则(如操作顺序)构成作出精确的假设。 我们的工作表明,神经网络不仅能够推断出其培训数据中隐含的结构化关系,而且还可以运用这种知识来指导个人含义的构成形成合成整体。