Image super-resolution (SR) techniques have been developing rapidly, benefiting from the invention of deep networks and its successive breakthroughs. However, it is acknowledged that deep learning and deep neural networks are difficult to interpret. SR networks inherit this mysterious nature and little works make attempt to understand them. In this paper, we perform attribution analysis of SR networks, which aims at finding the input pixels that strongly influence the SR results. We propose a novel attribution approach called local attribution map (LAM), which inherits the integral gradient method yet with two unique features. One is to use the blurred image as the baseline input, and the other is to adopt the progressive blurring function as the path function. Based on LAM, we show that: (1) SR networks with a wider range of involved input pixels could achieve better performance. (2) Attention networks and non-local networks extract features from a wider range of input pixels. (3) Comparing with the range that actually contributes, the receptive field is large enough for most deep networks. (4) For SR networks, textures with regular stripes or grids are more likely to be noticed, while complex semantics are difficult to utilize. Our work opens new directions for designing SR networks and interpreting low-level vision deep models.
翻译:图像超分辨率(SR)技术正在迅速发展,得益于深层网络的发明及其相继突破。然而,人们认识到深层学习和深神经网络很难解释。SR网络继承了这种神秘性质,很少有作品试图理解这些神秘性质。在本文中,我们对SR网络进行属性分析,目的是寻找对SR结果有重大影响的输入像素。我们提议了一个名为本地归属图(LAM)的新定位方法,它继承了整体梯度方法,但有两个独特的特点。一个是使用模糊图像作为基线输入,另一个是采用渐进模糊功能作为路径功能。我们以LAM为基础,显示:(1) 具有更广泛参与投入像素的SR网络可以取得更好的性能。(2) 关注网络和非本地网络从更广泛的投入像素中提取特征。(3) 与实际贡献的范围相比,可容纳的字段对于最深的网络来说足够大。(4) 对于SR网络来说,带有常规条纹或网格的文本更加容易被注意,而我们很难在深层次的网络中发现,而复杂的深层次的图像模型则难以利用。