Systematic generalization is the ability to combine known parts into novel meaning; an important aspect of efficient human learning, but a weakness of neural network learning. In this work, we investigate how two well-known modeling principles -- modularity and data augmentation -- affect systematic generalization of neural networks in grounded language learning. We analyze how large the vocabulary needs to be to achieve systematic generalization and how similar the augmented data needs to be to the problem at hand. Our findings show that even in the controlled setting of a synthetic benchmark, achieving systematic generalization remains very difficult. After training on an augmented dataset with almost forty times more adverbs than the original problem, a non-modular baseline is not able to systematically generalize to a novel combination of a known verb and adverb. When separating the task into cognitive processes like perception and navigation, a modular neural network is able to utilize the augmented data and generalize more systematically, achieving 70% and 40% exact match increase over state-of-the-art on two gSCAN tests that have not previously been improved. We hope that this work gives insight into the drivers of systematic generalization, and what we still need to improve for neural networks to learn more like humans do.
翻译:系统性的概括化是将已知部分整合为新含义的能力; 高效人类学习的一个重要方面, 但神经网络学习的一个弱点。 在这项工作中,我们调查两个众所周知的模型原则 -- -- 模块化和数据增强 -- -- 如何影响在基础语言学习中系统地普及神经网络。 我们分析词汇需要多大的词汇才能实现系统化的概括化,以及扩大的数据需要与手头的问题如何相似。 我们的研究结果显示,即使在合成基准的控制设置中,实现系统化的普及化也仍然非常困难。 在对比原始问题多近40倍的反动的扩大数据集进行培训之后,非模式基线无法系统化地将已知动词和动词组合成新颖的组合。 当将任务分为认知过程,如感知和导航时,模块神经网络能够利用强化的数据并更系统化,实现70%和40%的精确匹配率超过两个以前没有改进的GSCAN测试的状态。 我们希望这项工作能够让驱动者了解系统化的概括化和动词。 我们还需要改进什么来改进网络。