Convolutional neural network (CNN) accelerators are being widely used for their efficiency, but they require a large amount of memory, leading to the use of a slow and power consuming external memory. This paper exploits two schemes to reduce the required memory amount and ultimately to implement a CNN of reasonable performance only with on-chip memory of a practical device like a low-end FPGA. To reduce the memory amount of the intermediate data, a stream-based line-buffer architecture and a dataflow for the architecture are proposed instead of the conventional frame-based architecture, where the amount of the intermediate data memory is proportional to the square of the input image size. The architecture consists of layer-dedicated blocks operating in a pipelined way with the input and output streams. Each convolutional layer block has a line buffer storing just a few rows of input data. The sizes of the line buffers are proportional to the width of the input image, so the architecture requires less intermediate data storage than the conventional frame-based architecture, especially in the trend of getting larger input size in modern object detection CNNs. In addition to the reduced intermediate data storage, the weight memory is reduced by the accelerator-aware pruning. The experimental results show that a whole object detection CNN can be implemented even on a low-end FPGA without an external memory. Compared to previous accelerators with similar object detection accuracy, the proposed accelerator reaches much higher throughput even with less FPGA resources of LUTs, registers, and DSPs, showing much higher efficiency. The trained models and implemented bit files are available at https://github.com/HyeongjuKang/accelerator-aware-pruning and https://github.com/HyeongjuKang/aocstream.
翻译:电动神经网络加速器(CNN)正在被广泛用于提高效率,但它们需要大量记忆,从而导致使用缓慢和耗电的外部记忆。 本文利用两个方案来减少所需的内存量,并最终实施合理性能的CNNCN, 只在芯片上存储像低端 FPGA 这样的实用设备。 为了减少中间数据的存储量, 以流为基础的线缓冲架构和结构的数据流流流, 而不是传统的基于框架的结构, 中间数据存储量与输入图像的正方位成正比, 中间数据存储量甚至与输入图像的正方位成正方位。 本文利用两个方案来减少所需的内存数量, 并且利用输入和输出流流流流流流流流的管道运行方式运行层分层。 每个电层层的缓冲度只存储几行输入数据。 线缓冲的大小与输入图像的宽度成正比, 因此, 架构所需要的中间数据存储量比常规基基结构要少一些, 特别是在现代目标检测器的更大输入大小的趋势上, 。 此外, 也通过低的内存的内存数据存储器,, 将显示一个低的内存 。