Academic neural models for coreference resolution are typically trained on a single dataset (OntoNotes) and model improvements are then benchmarked on that dataset. However, real-world usages of coreference resolution models depend on the annotation guidelines and the domain of the target dataset, which often differ from those of OntoNotes. We aim to quantify transferability of coreference resolution models based on the number of annotated documents available in the target dataset. We examine five target datasets and find that continued training is consistently effective and especially beneficial when there are few target documents. We establish new benchmarks across several datasets, including state-of-the-art results on LitBank and PreCo.
翻译:用于共同参考分辨率的学术神经模型通常就单一数据集(OntoNotes)进行培训,然后以该数据集作为模型改进的基准,然而,共同参考分辨率模型的实际使用取决于说明准则和目标数据集的域,这往往不同于OntoNotes。我们的目标是根据目标数据集中附加说明的文件数量量化共同参考分辨率模型的可转让性。我们检查了五个目标数据集,发现在目标文件很少的情况下,持续培训始终有效,特别有益。我们为几个数据集制定了新的基准,包括LitBank和PreCo的最新结果。