Modern single image super-resolution (SISR) system based on convolutional neural networks (CNNs) achieves fancy performance while requires huge computational costs. The problem on feature redundancy is well studied in visual recognition task, but rarely discussed in SISR. Based on the observation that many features in SISR models are also similar to each other, we propose to use shift operation to generate the redundant features (i.e., Ghost features). Compared with depth-wise convolution which is not friendly to GPUs or NPUs, shift operation can bring practical inference acceleration for CNNs on common hardware. We analyze the benefits of shift operation for SISR and make the shift orientation learnable based on Gumbel-Softmax trick. For a given pre-trained model, we first cluster all filters in each convolutional layer to identify the intrinsic ones for generating intrinsic features. Ghost features will be derived by moving these intrinsic features along a specific orientation. The complete output features are constructed by concatenating the intrinsic and ghost features together. Extensive experiments on several benchmark models and datasets demonstrate that both the non-compact and lightweight SISR models embedded in our proposed module can achieve comparable performance to that of their baselines with large reduction of parameters, FLOPs and GPU latency. For instance, we reduce the parameters by 47%, FLOPs by 46% and GPU latency by 41% of EDSR x2 network without significant performance degradation.
翻译:以进化神经网络(CNNs)为基础的现代单一图像超分辨率系统(SISR)在需要大量计算成本的情况下可以取得高性能,而需要大量计算成本。关于功能冗余的问题在视觉识别任务中得到了很好地研究,但在SISR中很少讨论。基于SISSR模型中的许多特征彼此相似的观察,我们建议使用转换操作来产生冗余特征(如幽灵特征)。与不友好于 GPUs 或 NPUs 的深层次变动相比,转换操作可以为CNN在通用硬件上带来实际的加速。我们分析了SISSR的变换操作的好处,并使根据 Gumbel- Softmax 的技巧学习变换方向。对于给定的预训练模型,我们首先将每个变动层的所有过滤器组合起来,以辨别产生内在特征的内在特征(如幽灵特征)。 与这些内在特性相适应的变异的变异性能特征是通过将内置的xOP2 和鬼性能参数结合在一起来构建完整的输出特征。在几个基准模型和数据集中进行广泛的变异性模型中,通过比较性化的GOPLBISL 和光模型可以降低的系统模型,通过我们的大规模性能基准模型和光性能基准模型来降低性能模型,从而降低性能模型。