This paper presents a technical report of our submission to the 4th task of SemEval-2021, titled: Reading Comprehension of Abstract Meaning. In this task, we want to predict the correct answer based on a question given a context. Usually, contexts are very lengthy and require a large receptive field from the model. Thus, common contextualized language models like BERT miss fine representation and performance due to the limited capacity of the input tokens. To tackle this problem, we used the Longformer model to better process the sequences. Furthermore, we utilized the method proposed in the Longformer benchmark on Wikihop dataset which improved the accuracy on our task data from 23.01% and 22.95% achieved by the baselines for subtask 1 and 2, respectively, to 70.30% and 64.38%.
翻译：本文件介绍了我们提交SemEval-2021第4项任务的文件的技术报告,题为:阅读摘要含义的理解。在这项任务中,我们希望根据一个给定的上下文预测正确的答案。通常,背景非常冗长,需要从模型中获取一个大范围的可接受字段。因此,像BERT这样的通用背景化语言模型由于输入符号能力有限而错失了精细的表述和性能。为了解决这一问题,我们使用了长式模型来更好地处理序列。此外,我们使用了Wikihop数据集长式基准中建议的方法,该方法提高了我们任务数据的准确性,分别从23.01%和22.95%提高到70.30%和64.38%,次任务1和2的基线分别达到23.01%和22.95%。
相关内容
微信扫码咨询专知VIP会员