This paper describes our approach to the multi-modal fact verification (FACTIFY) challenge at AAAI2023. In recent years, with the widespread use of social media, fake news can spread rapidly and negatively impact social security. Automatic claim verification becomes more and more crucial to combat fake news. In fact verification involving multiple modal data, there should be a structural coherence between claim and document. Therefore, we proposed a structure coherence-based multi-modal fact verification scheme to classify fake news. Our structure coherence includes the following four aspects: sentence length, vocabulary similarity, semantic similarity, and image similarity. Specifically, CLIP and Sentence BERT are combined to extract text features, and ResNet50 is used to extract image features. In addition, we also extract the length of the text as well as the lexical similarity. Then the features were concatenated and passed through the random forest classifier. Finally, our weighted average F1 score has reached 0.8079, achieving 2nd place in FACTIFY2.
翻译:本文描述了我们对AAAI2023号多模式事实核查(FACTIFY)挑战的处理方法。近年来,随着社交媒体的广泛使用,假新闻可以迅速传播,对社会保障产生消极影响。自动索赔核查对打击假新闻越来越重要。事实上,涉及多种模式数据的核查,在索赔和文件之间应当有结构性的一致性。因此,我们提出了一个基于结构的基于一致性的多模式事实核查计划,对假新闻进行分类。我们的结构一致性包括以下四个方面:句号长度、词汇相似性、语义相似性和图像相似性。具体来说,CLIP和BERT组合在一起,提取文本功能,ResNet50被用于提取图像特征。此外,我们还提取文本的长度以及词汇相似性。然后,这些特征被组合并传递到随机森林分类器中。最后,我们的加权平均F1分达到了0.8079,在FACTIFY2中达到了第2位。</s>