We propose the framework of dual convexified convolutional neural networks (DCCNNs). In this framework, we first introduce a primal learning problem motivated from convexified convolutional neural networks (CCNNs), and then construct the dual convex training program through careful analysis of the Karush-Kuhn-Tucker (KKT) conditions and Fenchel conjugates. Our approach reduces the memory overhead of constructing a large kernel matrix and eliminates the ambiguity of factorizing the matrix. Due to the low-rank structure in CCNNs and the related subdifferential of nuclear norms, there is no closed-form expression to recover the primal solution from the dual solution. To overcome this, we propose a highly novel weight recovery algorithm, which takes the dual solution and the kernel information as the input, and recovers the linear and convolutional weights of a CCNN. Furthermore, our recovery algorithm exploits the low-rank structure and imposes a small number of filters indirectly, which reduces the parameter size. As a result, DCCNNs inherit all the statistical benefits of CCNNs, while enjoying a more formal and efficient workflow.
翻译:我们提议建立双重凝聚神经神经网络(DCCNNs)框架。在这个框架内,我们首先引入了由混集神经网络(CCNNs)引发的原始学习问题,然后通过仔细分析Karush-Kuhn-Tucker(KKT)条件和Fenchel conjugates(Fenchel)来构建双重凝聚神经网络(DCCNNs),然后通过仔细分析Karush-Kuhn-Tucker(KKTTT)条件和Fenchel conjugates(Fenchel)来构建双重凝聚神经网络(DCCNNs)。我们的方法减少了建造大型内脏矩阵的记忆管理,消除了矩阵因素的模糊性。由于CCNNes的低级别结构和相关的核规范的次偏差,我们没有封闭式的表达方式来从双重解决方案中恢复原始的原始解决方案。为了克服这一点,我们提出了一种非常新颖的重量回收算法,把双重解决方案和内脏信息作为投入,并恢复CCNs的线性和革命权重。此外,我们的恢复算算法利用了低级结构结构,并间接强加了少量过滤器,从而降低了参数大小。结果,DCCNNs继承了CCNs的所有统计利益,同时享有了CCNs具有更正式和高效的工作流程。