We present a generic method for recurrently using the same parameters for many different convolution layers to build a deep network. Specifically, for a network, we create a recurrent parameter generator (RPG), from which the parameters of each convolution layer are generated. Though using recurrent models to build a deep convolutional neural network (CNN) is not entirely new, our method achieves significant performance gain compared to the existing works. We demonstrate how to build a one-layer neural network to achieve similar performance compared to other traditional CNN models on various applications and datasets. Such a method allows us to build an arbitrarily complex neural network with any amount of parameters. For example, we build a ResNet34 with model parameters reduced by more than $400$ times, which still achieves $41.6\%$ ImageNet top-1 accuracy. Furthermore, we demonstrate the RPG can be applied at different scales, such as layers, blocks, or even sub-networks. Specifically, we use the RPG to build a ResNet18 network with the number of weights equivalent to one convolutional layer of a conventional ResNet and show this model can achieve $67.2\%$ ImageNet top-1 accuracy. The proposed method can be viewed as an inverse approach to model compression. Rather than removing the unused parameters from a large model, it aims to squeeze more information into a small number of parameters. Extensive experiment results are provided to demonstrate the power of the proposed recurrent parameter generator.
翻译:我们提出一种通用方法,用于经常使用许多不同变迁层的相同参数来建立深网络。 具体地说, 对于一个网络, 我们创建了一个经常的参数生成器( RPG), 由此产生每个变迁层的参数。 尽管我们使用反复的模型来建立深变动神经网络(CNN)并不是完全新颖的, 我们的方法与现有工程相比取得了显著的性能收益。 我们展示了如何建立一个单层神经网络, 以取得与各种应用和数据集方面其他传统的CNN模型类似的性能。 这种方法使我们能够建立一个任意的复杂神经网络。 例如, 我们建造一个 ResNet34, 其模型参数减少400美元以上, 但仍能达到 41.6 $ 图像网络顶层-1 的精确度。 此外, 我们展示了火箭可以在不同尺度上应用, 如层、 区块, 甚至次网络。 具体地说, 我们用RPG 来建立一个 ResNet 网络 网络 网络 网络 网络, 其重量相当于 常规 ResNet 层 的 一个革命层层, 和 显示这个模型可以实现67.2 $$ 图像 网络 的经常 的 值, 至 底压值 的精确度 。