Although high-performance deep neural networks are in high demand in edge environments, computation resources are strictly limited in edge devices, and light-weight neural network techniques, such as Depthwise Separable Convolution (DSC), have been developed. ResNet is one of conventional deep neural network models that stack a lot of layers and parameters for a higher accuracy. To reduce the parameter size of ResNet, by utilizing a similarity to ODE (Ordinary Differential Equation), Neural ODE repeatedly uses most of weight parameters instead of having a lot of different parameters. Thus, Neural ODE becomes significantly small compared to that of ResNet so that it can be implemented in resource-limited edge devices. In this paper, a combination of Neural ODE and DSC, called dsODENet, is designed and implemented for FPGAs (Field-Programmable Gate Arrays). dsODENet is then applied to edge domain adaptation as a practical use case and evaluated with image classification datasets. It is implemented on Xilinx ZCU104 board and evaluated in terms of domain adaptation accuracy, training speed, FPGA resource utilization, and speedup rate compared to a software execution. The results demonstrate that dsODENet is comparable to or slightly better than our baseline Neural ODE implementation in terms of domain adaptation accuracy, while the total parameter size without pre- and post-processing layers is reduced by 54.2% to 79.8%. The FPGA implementation accelerates the prediction tasks by 27.9 times faster than a software implementation.
翻译:虽然高性能深神经网络在边缘环境中的需求很高,但计算资源在边缘设备中严格有限,并且已经开发了轻量型神经网络技术,例如深度分解变异(DSC),ResNet是传统的深心神经网络模型之一,它堆积了许多层次和参数,以便更加精确。为了缩小ResNet的参数大小,利用与ODE(常规差异方程式)的相似性,Neal ODE反复使用大部分重量参数,而不是许多不同的参数。因此,与ResNet相比,神经系统变得非常小,因此可以在资源有限的边缘设备中使用轻量的神经网络技术。在本文件中,Neural ODE和DSC(称为DsODENet)的组合为常规神经深心网络模型和参数,为FPGA(外地可Prographramable Gate Arrays)设计和实施。DsODENet随后应用边缘域适应作为实际使用的案例,并用图像分类数据集进行评估。在Xlinx ZCUU104 104 董事会上实施,从域适应总准确性、培训速度、FDA(54GA)的精确度时间、FDER资源利用率和速度比实际执行率略地域域范围的比SDRODFDFDE的精确度要小化,在比SDRDFDFDR的精确度执行率的精确度上进行测试。