Along with the rapid progress of visual tracking, existing benchmarks become less informative due to redundancy of samples and weak discrimination between current trackers, making evaluations on all datasets extremely time-consuming. Thus, a small and informative benchmark, which covers all typical challenging scenarios to facilitate assessing the tracker performance, is of great interest. In this work, we develop a principled way to construct a small and informative tracking benchmark (ITB) with 7% out of 1.2 M frames of existing and newly collected datasets, which enables efficient evaluation while ensuring effectiveness. Specifically, we first design a quality assessment mechanism to select the most informative sequences from existing benchmarks taking into account 1) challenging level, 2) discriminative strength, 3) and density of appearance variations. Furthermore, we collect additional sequences to ensure the diversity and balance of tracking scenarios, leading to a total of 20 sequences for each scenario. By analyzing the results of 15 state-of-the-art trackers re-trained on the same data, we determine the effective methods for robust tracking under each scenario and demonstrate new challenges for future research direction in this field.
翻译:随着目视跟踪的快速进展,现有基准由于样本冗余和当前跟踪者之间歧视微弱而信息量减少,使得对所有数据集的评价极其耗时,因此,一个内容翔实的小型基准,涵盖所有典型的具有挑战性的情景,以便利对跟踪者业绩进行评估,很有意义。在这项工作中,我们制定了一个原则性方法,在1.2个现有和新收集的数据集的M框架中建立一个小的信息量跟踪基准(ITB),7%的跟踪基准能够进行有效评估,同时确保有效性。具体地说,我们首先设计一个质量评估机制,从现有基准中选择信息最丰富的序列,同时考虑到:(1) 具有挑战性的水平,(2) 区分强度,(3) 外观变化的密度。此外,我们收集了更多的序列,以确保跟踪情景的多样性和平衡性,导致每个情景总共20个序列。通过分析15个最先进的跟踪者对同一数据进行再培训的结果,我们确定在每种情景下进行强有力跟踪的有效方法,并展示该领域未来研究方向的新挑战。