Internet memes have become a dominant method of communication; at the same time, however, they are also increasingly being used to advocate extremism and foster derogatory beliefs. Nonetheless, we do not have a firm understanding as to which perceptual aspects of memes cause this phenomenon. In this work, we assess the efficacy of current state-of-the-art multimodal machine learning models toward hateful meme detection, and in particular with respect to their generalizability across platforms. We use two benchmark datasets comprising 12,140 and 10,567 images from 4chan's "Politically Incorrect" board (/pol/) and Facebook's Hateful Memes Challenge dataset to train the competition's top-ranking machine learning models for the discovery of the most prominent features that distinguish viral hateful memes from benign ones. We conduct three experiments to determine the importance of multimodality on classification performance, the influential capacity of fringe Web communities on mainstream social platforms and vice versa, and the models' learning transferability on 4chan memes. Our experiments show that memes' image characteristics provide a greater wealth of information than its textual content. We also find that current systems developed for online detection of hate speech in memes necessitate further concentration on its visual elements to improve their interpretation of underlying cultural connotations, implying that multimodal models fail to adequately grasp the intricacies of hate speech in memes and generalize across social media platforms.
翻译:互联网模式已成为一种占主导地位的通信方法;然而,与此同时,它们也越来越多地被用于鼓吹极端主义和培养贬低信仰。然而,我们对于Memes的观念方面导致这种现象的原因并不十分了解。在这项工作中,我们评估了目前最先进的多式联运机器学习模型的功效,这些模型有助于进行仇恨的Memme检测,特别是其在整个平台上的通用性能。我们使用两个基准数据集,其中包括来自4chan的“政治不正确”董事会(/Pol/)和Facebook的“令人憎恶的Memes挑战”数据集的12,140和10,567图像。我们的实验显示,Memes的图像特性比其文本内容提供了更多的信息。我们发现竞争者最高级的机器学习模型,以发现最突出的特征,这些特征区别于病毒仇恨与良性Memes。我们进行了三次实验,以确定在主流社会平台上和反之边际网络社群的影响力的重要性,以及模型在4champmemes上学习可转移性。我们的实验显示,Memes的图像特征特征提供了比其文字内容内容更加丰富的信息。我们发现,在视觉模型中也有必要使当前系统在视觉分析其基础上改进了。