We present LogiGAN, an unsupervised adversarial pre-training framework for improving logical reasoning abilities of language models. Upon automatic identifying logical reasoning phenomena in massive text corpus via detection heuristics, we train language models to predict the masked-out logical statements. Inspired by the facilitation effect of reflective thinking in human learning, we analogically simulate the learning-thinking process with an adversarial Generator-Verifier architecture to assist logic learning. LogiGAN implements a novel sequential GAN approach that (a) circumvents the non-differentiable challenge of the sequential GAN by leveraging the Generator as a sentence-level generative likelihood scorer with a learning objective of reaching scoring consensus with the Verifier; (b) is computationally feasible for large-scale pre-training with arbitrary target length. Both base and large size language models pre-trained with LogiGAN demonstrate obvious performance improvement on 12 datasets requiring general reasoning abilities, revealing the fundamental role of logic in broad reasoning, as well as the effectiveness of LogiGAN. Ablation studies on LogiGAN components reveal the relative orthogonality between linguistic and logic abilities and suggest that reflective thinking's facilitation effect might also generalize to machine learning.
翻译:我们提出LogiGAN,这是提高语言模型逻辑推理能力的不受监督的对抗性初步训练框架,用于提高语言模型的逻辑推理能力。在通过探测超光速,自动确定大量文本库中逻辑推理现象的逻辑推理现象时,我们培训语言模型,以预测隐藏的逻辑说明;在人类学习中反思思维的促进作用的启发下,我们模拟学习思维过程,用一个对抗性发电机-变异器结构模拟,以协助逻辑学习。LogiGAN采用一种新型的顺序性GAN方法,即(a) 利用发电机作为句级基因级概率分数,以学习目标为目的,与验证者达成评分共识,从而绕过顺序GAN的不可区分的挑战。 (b) 用于任意目标长度的大规模预培训,在计算上都是可行的。与LogiGAN预先培训的基地和大型语言模型都表明12个数据集的性能明显改进,需要一般推理能力,揭示逻辑在广义推理中的基本作用,以及LogiGAN的有效性。LogiGAN的调整研究显示LogiGAN组成部分显示LogiGAN的相对或高度理解能力,并且也反映了语言和逻辑解释机的思维能力之间的一般学习效果。