Traditional object detectors employ the dense paradigm of scanning over locations and scales in an image. The recent query-based object detectors break this convention by decoding image features with a set of learnable queries. However, this paradigm still suffers from slow convergence, limited performance, and design complexity of extra networks between backbone and decoder. In this paper, we find that the key to these issues is the adaptability of decoders for casting queries to varying objects. Accordingly, we propose a fast-converging query-based detector, named AdaMixer, by improving the adaptability of query-based decoding processes in two aspects. First, each query adaptively samples features over space and scales based on estimated offsets, which allows AdaMixer to efficiently attend to the coherent regions of objects. Then, we dynamically decode these sampled features with an adaptive MLP-Mixer under the guidance of each query. Thanks to these two critical designs, AdaMixer enjoys architectural simplicity without requiring dense attentional encoders or explicit pyramid networks. On the challenging MS COCO benchmark, AdaMixer with ResNet-50 as the backbone, with 12 training epochs, reaches up to 45.0 AP on the validation set along with 27.9 APs in detecting small objects. With the longer training scheme, AdaMixer with ResNeXt-101-DCN and Swin-S reaches 49.5 and 51.3 AP. Our work sheds light on a simple, accurate, and fast converging architecture for query-based object detectors. The code is made available at https://github.com/MCG-NJU/AdaMixer
翻译:传统天体探测器采用了对图像中的位置和比例进行扫描的密集模式。 最近的基于查询的天体探测器破解了这个公约, 解码了图像特征, 并提供了一组可学习的查询。 但是, 这个模式仍然因主干线和解码器之间额外网络的趋同缓慢、 性能有限和设计复杂而受到影响。 在本文中, 我们发现这些问题的关键在于解码器对不同对象进行查询的适应性。 因此, 我们提出一个快速趋同的基于查询的探测器, 名为AdaMixer, 提高基于查询的解码过程在两个方面的适应性。 首先, 以估计的抵消为基础, 每对空间和尺度上的适应性样本特征进行解码处理, 使AdaMixer能够高效地关注连贯的物体区域。 然后, 我们用适应性 MLP- Mixer 来动态解码解析这些特征。 由于这两个关键设计, AdaMixer在不需要密集的注意编码或明确的金字塔网络中, 具有挑战性的 MSCO 基准, Ada- mixer, Ad- mix 和 Re- 503 以及 Re- sy- hold- hold- hold- hold- hold- hold- silal- sal- sal- silveal- sal- silveal- sal- sal- sal- sal- six 和12 trade- semstr