Children acquiring English make systematic errors on subject control sentences even after they have reached near-adult competence (C. Chomsky, 1969), possibly due to heuristics based on semantic roles (Maratsos, 1974). Given the advanced fluency of large generative language models, we ask whether model outputs are consistent with these heuristics, and to what degree different models are consistent with each other. We find that models can be categorized by behavior into three separate groups, with broad differences between the groups. The outputs of models in the largest group are consistent with positional heuristics that succeed on subject control but fail on object control. This result is surprising, given that object control is orders of magnitude more frequent in the text data used to train such models. We examine to what degree the models are sensitive to prompting with agent-patient information, finding that raising the salience of agent and patient relations results in significant changes in the outputs of most models. Based on this observation, we leverage an existing dataset of semantic proto-role annotations (White, et al. 2020) to explore the connections between control and labeling event participants with properties typically associated with agents and patients.
翻译:获得英语的儿童甚至在达到接近成年能力后(C.Chomsky,1969年),也可能由于基于语义作用的超自然学(Maratsos,1974年)。鉴于大型基因化语言模型的高度流利,我们询问模型产出是否与这些超常性语言模型一致,不同模型在多大程度上彼此一致。我们发现模型可以按行为分为三个不同的类别,各组之间差别很大。最大组的模型产出与在对象控制上成功但物体控制失败的定位超常一致。这一结果令人吃惊,因为在用于培训这些模型的文本数据中,对象控制是更频繁的级。我们研究模型在多大程度上敏感地了解与代理-病人信息的迅速性,发现提高代理和病人关系的显著性能导致大多数模型产出的重大变化。基于这一观察,我们利用了现有一套语系原功能说明(White, et al. 2020年)来探索控制事件参与者与典型的代理人和病人之间的关联。