Evidence data for automated fact-checking (AFC) can be in multiple modalities such as text, tables, images, audio, or video. While there is increasing interest in using images for AFC, previous works mostly focus on detecting manipulated or fake images. We propose a novel task, chart-based fact-checking, and introduce ChartBERT as the first model for AFC against chart evidence. ChartBERT leverages textual, structural and visual information of charts to determine the veracity of textual claims. For evaluation, we create ChartFC, a new dataset of 15, 886 charts. We systematically evaluate 75 different vision-language (VL) baselines and show that ChartBERT outperforms VL models, achieving 63.8% accuracy. Our results suggest that the task is complex yet feasible, with many challenges ahead.
翻译:自动事实核对(AFC)的证据数据可以采用多种模式,如文本、表格、图像、音频或视频等。虽然人们越来越有兴趣为AFC使用图像,但以前的工作主要侧重于探测被操纵或伪造的图像。我们提议了一项新任务,以图表为基础进行事实核对,并采用ChartBERT作为AFC与图表证据相对的第一个模型。ChapBERT利用图表的文字、结构和视觉信息来确定文本索赔的真实性。为了评估,我们创建了ChartFC,这是一个由15,886个图表组成的新数据集。我们系统地评估了75个不同的视觉语言(VL)基线,并显示ChartBERT优于VL模型,实现了63.8%的准确性。我们的结果表明,这项任务既复杂,又可行,未来有许多挑战。