Multi-hop Machine reading comprehension is a challenging task with aim of answering a question based on disjoint pieces of information across the different passages. The evaluation metrics and datasets are a vital part of multi-hop MRC because it is not possible to train and evaluate models without them, also, the proposed challenges by datasets often are an important motivation for improving the existing models. Due to increasing attention to this field, it is necessary and worth reviewing them in detail. This study aims to present a comprehensive survey on recent advances in multi-hop MRC evaluation metrics and datasets. In this regard, first, the multi-hop MRC problem definition will be presented, then the evaluation metrics based on their multi-hop aspect will be investigated. Also, 15 multi-hop datasets have been reviewed in detail from 2017 to 2022, and a comprehensive analysis has been prepared at the end. Finally, open issues in this field have been discussed.
翻译：多跳机器阅读理解是一项具有挑战性的任务,目的是回答一个基于不同段落信息脱节的问题,评价指标和数据集是多跳机器MRC的一个重要部分,因为没有这些指标和数据集就不可能培训和评价模型,此外,由数据集提出的挑战往往也是改进现有模型的重要动机。由于日益关注这一领域,有必要也值得详细审查它们。本研究的目的是就多跳MRC评价指标和数据集方面的最新进展进行一次全面调查。在这方面,首先将提出多跳MRC问题定义,然后将调查基于多跳模型方面的评价指标。此外,2017年至2022年对15个多跳数据集进行了详细审查,并在最后进行了全面分析。最后,讨论了该领域的公开问题。
相关内容
微信扫码咨询专知VIP会员