This paper presents a comprehensive evaluation of instance segmentation models with respect to real-world image corruptions and out-of-domain image collections, e.g. datasets collected with different set-ups than the training datasets the models learned from. The out-of-domain image evaluation shows the generalization capability of models, an essential aspect of real-world applications, and an extensively studied topic of domain adaptation. These presented robustness and generalization evaluations are important when designing instance segmentation models for real-world applications and picking an off-the-shelf pretrained model to directly use for the task at hand. Specifically, this benchmark study includes state-of-the-art network architectures, network backbones, normalization layers, models trained starting from scratch or ImageNet pretrained networks, and the effect of multi-task training on robustness and generalization. Through this study, we gain several insights e.g. we find that normalization layers play an essential role in robustness, ImageNet pretraining does not help the robustness and the generalization of models, excluding JPEG corruption, and network backbones and copy-paste augmentations affect robustness significantly.
翻译:本文全面评价了真实世界图像腐败和外部图像收集方面实例分类模型的综合性评价,例如,与培训数据集相比,用不同组别收集的数据集,而不是从中学习的模型。外部图像评价显示了模型的概括性能力,这是现实世界应用的一个基本方面,也是广泛研究的领域适应专题。这些文件展示了稳健性和概括性评价,在设计真实世界应用实例分类模型和挑选直接用于当前任务的现成预先培训模型时非常重要。具体而言,本基准研究包括最新网络结构、网络主干线、正常化层、从抓起或图像网络先行培训的模式,以及多任务培训对稳健性和概括性的影响。通过这项研究,我们获得了若干见解,例如,我们认为正常化层在稳健性方面起着关键作用,图像网络前期培训无助于模型的稳健性和普遍化,不包括JEG腐败,网络主干线和复制版加固性增强性将严重影响稳性。