The last decade has witnessed the breakthrough of deep neural networks (DNNs) in many fields. With the increasing depth of DNNs, hundreds of millions of multiply-and-accumulate (MAC) operations need to be executed. To accelerate such operations efficiently, analog in-memory computing platforms based on emerging devices, e.g., resistive RAM (RRAM), have been introduced. These acceleration platforms rely on analog properties of the devices and thus suffer from process variations and noise. Consequently, weights in neural networks configured into these platforms can deviate from the expected values, which may lead to feature errors and a significant degradation of inference accuracy. To address this issue, in this paper, we propose a framework to enhance the robustness of neural networks under variations and noise. First, a modified Lipschitz constant regularization is proposed during neural network training to suppress the amplification of errors propagated through network layers. Afterwards, error compensation is introduced at necessary locations determined by reinforcement learning to rescue the feature maps with remaining errors. Experimental results demonstrate that inference accuracy of neural networks can be recovered from as low as 1.69% under variations and noise back to more than 95% of their original accuracy, while the training and hardware cost are negligible.
翻译:过去十年中,许多领域的深神经网络(DNN)取得了突破。随着DNN的深度日益加深,需要执行数以亿亿计的倍增和累积(MAC)操作。为了高效率地加速这类操作,采用了基于新兴装置的模拟模拟内模计算平台,例如耐抗性RAM(RRAM),这些加速平台依靠设备的模拟特性,因而受到过程变异和噪音的影响。因此,在这些平台中配置的神经网络重量可能偏离预期值,这可能导致特征错误和推断准确性严重退化。为了解决这一问题,我们在本文件中提议了一个框架,以加强在变异和噪音下神经网络网络的稳健性。首先,在神经网络培训期间建议修改Lipschitz的常态规范,以抑制通过网络层传播错误的放大。随后,在通过强化学习而确定的必要地点引入错误补偿办法,以便用其余错误来挽救地貌地图。实验结果显示,原始神经网络的推断性准确性可以从1.69的低点恢复过来,而硬件的精确度则在硬件变异和噪声中恢复到更低的95%。