Language models have achieved impressive results in natural language processing tasks, but their ability to perform symbolic operations and arithmetic operations, remains limited, which attribute to their learn the rules implicitly from data. We explore how to incorporate compiled neural networks (CoNNs) which weight is specially designed, into the architecture of language models to enable the language model trained by gradient to obtain fully rule comprehension ability. The incorporation of compiled neural networks offers a promising direction for improving the performance of language models on compound tasks, particularly in areas that require a deeper comprehension of abstract rules beyond recognizing patterns in training data. Our method, which call "Neural Comprehension", helps language models achieve absolute accuracy in symbolic operations, thereby enhancing their ability for rule reasoning, symbolic reasoning, and arithmetic reasoning. Our code is publicly available at: \url{https://github.com/WENGSYX/Neural-Comprehension}.
翻译:语言模型在自然语言处理任务中取得了令人瞩目的结果,但它们执行符号操作和算术操作的能力仍然受到限制,这归因于它们从数据中隐式地学习规则。我们探索了如何将专为权重设计的编译神经网络(CoNN)纳入语言模型的架构中,从而使通过梯度训练的语言模型具备完全的规则理解能力。编译神经网络的纳入为提高语言模型在复合任务中的表现提供了有前途的方向,特别是在需要超出识别训练数据中的模式的抽象规则的更深层次理解的领域。我们的方法称为“神经理解”,帮助语言模型在符号运算方面实现绝对精度,从而增强其规则推理、符号推理和算术推理的能力。我们的代码公开在 \url{https://github.com/WENGSYX/Neural-Comprehension}.