As tools for content editing mature, and artificial intelligence (AI) based algorithms for synthesizing media grow, the presence of manipulated content across online media is increasing. This phenomenon causes the spread of misinformation, creating a greater need to distinguish between "real" and "manipulated" content. To this end, we present VideoSham, a dataset consisting of 826 videos (413 real and 413 manipulated). Many of the existing deepfake datasets focus exclusively on two types of facial manipulations -- swapping with a different subject's face or altering the existing face. VideoSham, on the other hand, contains more diverse, context-rich, and human-centric, high-resolution videos manipulated using a combination of 6 different spatial and temporal attacks. Our analysis shows that state-of-the-art manipulation detection algorithms only work for a few specific attacks and do not scale well on VideoSham. We performed a user study on Amazon Mechanical Turk with 1200 participants to understand if they can differentiate between the real and manipulated videos in VideoSham. Finally, we dig deeper into the strengths and weaknesses of performances by humans and SOTA-algorithms to identify gaps that need to be filled with better AI algorithms.
翻译:随着内容编辑工具的成熟和基于人工智能(AI)的合成媒体工具的成熟和人工智能(AI)的算法的增加,在线媒体中受操纵的内容的存在正在增加。这一现象导致错误信息的传播,造成对“真实”和“管理”内容更需要进行区分。为此,我们介绍了由826个视频(413个真实和413个操作过的视频)组成的数据集“视频Sham”。许多现有的深假数据集专门侧重于两类面部操控 — — 与不同对象的面部互换或改变现有面部。另一方面,视频Sham包含更多多样化、环境丰富和以人为中心、高分辨率的视频,使用六种不同的时空攻击组合进行操纵。我们的分析显示,最新最先进的操纵检测算法只适用于少数特定袭击,而不是在视频Sham上进行放大。我们进行了亚马逊机械土耳其的用户研究,有1200名参与者,以了解他们能否区分视频Sham的真面和被操纵的视频。最后,我们更深入地挖掘了人类和SOTA算法的优缺点。我们需要更好地填补AIALs。