Commonsense knowledge can be leveraged for identifying causal relations in text. In this work, we verbalize triples in ATOMIC2020, a wide coverage commonsense reasoning knowledge graph, to natural language text and continually pretrain a BERT pretrained language model. We evaluate the resulting model on answering commonsense reasoning questions. Our results show that a continually pretrained language model augmented with commonsense reasoning knowledge outperforms our baseline on two commonsense causal reasoning benchmarks, COPA and BCOPA-CE, without additional improvement on the base model or using quality-enhanced data for fine-tuning.
翻译:在这项工作中,我们在ATOMIC2020中,用广泛覆盖的常识推理知识图,即广泛覆盖的常识推理知识图,对自然语言文本进行三番口述,并不断对BERT预先培训的语言模式进行预先培训。我们评估了由此产生的回答常识推理问题的模式。我们的结果显示,不断经过训练的语言模式,加上常识推理知识,超过了我们关于两个常识推理基准(COPA和BCOPA-CE)的基线,而没有在基准模型上作出进一步的改进,也没有使用质量强化数据进行微调。