In the past few years, cross-modal image-text retrieval (ITR) has experienced increased interest in the research community due to its excellent research value and broad real-world application. It is designed for the scenarios where the queries are from one modality and the retrieval galleries from another modality. This paper presents a comprehensive and up-to-date survey on the ITR approaches from four perspectives. By dissecting an ITR system into two processes: feature extraction and feature alignment, we summarize the recent advance of the ITR approaches from these two perspectives. On top of this, the efficiency-focused study on the ITR system is introduced as the third perspective. To keep pace with the times, we also provide a pioneering overview of the cross-modal pre-training ITR approaches as the fourth perspective. Finally, we outline the common benchmark datasets and valuation metric for ITR, and conduct the accuracy comparison among the representative ITR approaches. Some critical yet less studied issues are discussed at the end of the paper.
翻译:过去几年来,跨模式图像-文字检索(ITR)由于具有出色的研究价值和广泛的现实应用,对研究界的兴趣日益浓厚,因为其研究价值很高,而且具有广泛的现实应用性;它针对的是从一种模式和从另一种模式的检索画廊查询的情景;本文件从四个角度对ITR方法进行了全面的最新调查;通过将ITR系统分为两个进程:特征提取和特征协调,我们从这两个角度总结了IRT方法的最新进展;此外,关于IRT系统的效率重点研究作为第三个角度加以介绍;为了跟上时代的步伐,我们还从第四个角度对跨模式培训前的ITR方法进行开拓性概述;最后,我们概述了ITR的共同基准数据集和估价指标,并在具有代表性的 ITR方法之间进行精确比较;在文件结尾部分讨论一些研究较少的关键问题。