Retrieving target videos based on text descriptions is a task of great practical value and has received increasing attention over the past few years. In this paper, we focus on the less-studied setting of multi-query video retrieval, where multiple queries are provided to the model for searching over the video archive. We first show that the multi-query retrieval task is more pragmatic and representative of real-world use cases and better evaluates retrieval capabilities of current models, thereby deserving of further investigation alongside the more prevalent single-query retrieval setup. We then propose several new methods for leveraging multiple queries at training time to improve over simply combining similarity outputs of multiple queries from regular single-query trained models. Our models consistently outperform several competitive baselines over three different datasets. For instance, Recall@1 can be improved by 4.7 points on MSR-VTT, 4.1 points on MSVD and 11.7 points on VATEX over a strong baseline built on the state-of-the-art CLIP4Clip model. We believe further modeling efforts will bring new insights to this direction and spark new systems that perform better in real-world video retrieval applications. Code is available at https://github.com/princetonvisualai/MQVR.
翻译:根据文字描述检索目标视频是一项非常实际的工作,在过去几年中,我们越来越关注基于文本描述的一项任务。在本文中,我们注重研究较少的多查询视频检索的多查询设置,为视频档案搜索模型提供多种查询。我们首先显示,多查询检索任务更加务实,更能代表现实世界使用的案件,并更好地评价当前模型的检索能力,因此值得与更为普遍的单一查询检索设置一道进行进一步调查。然后我们提出几种新的方法,在培训时间利用多个查询,以改进仅将常规单一查询培训模型的多个查询的类似产出合并在一起的简单做法。我们的模型始终超越了三个不同数据集的若干竞争性基线。例如,在MSR-VTT、MSVD的4.1点和VATEX的11.7点上,可以在基于最新艺术CLIP4Clip 模型的强基线上进行改进。我们认为进一步的建模工作将给这个方向带来新的洞察力,并激发新的系统,在真实世界/Rbasimalimalimalimaling Q。