基于稀疏编码模型的深层学习神经网络

项目名称： 基于稀疏编码模型的深层学习神经网络

项目编号： No.61273023

项目类型： 面上项目

立项/批准年度： 2013

项目学科： 自动化技术、计算机技术

项目作者： 胡晓林

作者单位： 清华大学

项目金额： 61万元

中文摘要： 深层学习和稀疏编码在解释大脑的工作机理、挖掘数据的结构信息、抽取用于模式识别的特征等方面发挥着重要作用。目前二者的结合备受关注，但结合方式主要是在现有的深层学习网络基础上进行修改，增加稀疏性约束。这种方式虽然简单直接，但无法克服现有深层学习网络学习过程慢、参数调整复杂的固有缺陷。本项目从另一个角度出发，提出直接以稀疏编码模型为基础构建深层学习网络。拟研究的稀疏编码模型包括单层和双层模型，构建的深层学习网络将既有基于单层模型的，又有基于双层模型的，还有基于单、双层模型混合的。一方面，通过理论分析建立基于稀疏编码模型的深层学习网络的理论基础，另一方面，通过实验对比发明一系列适合实际应用问题的深层学习网络。鉴于稀疏编码模型简洁的结构和良好的算法性能，它们与深层学习网络结合的这种方式不仅更加自然，还有助于克服现有深层学习网络的上述固有缺陷，并有望在计算神经科学领域和模式识别相关领域中发挥重要作用。

中文关键词： 深度学习；稀疏编码；神经科学；神经网络；反馈连接

英文摘要： Deep learning and sparse coding play important roles in explaining the functional mechanism of the brain, mining the structural information of data and extracting features for pattern recognition. In these years, integration of the two models has attracted much interest. The main idea for the integration is adding sparsity constraints to the existing deep learning networks (DLNs). This method, though simple enough, cannot circumvent the inherent drawbacks of the existing DLNs including the slow learning process and troublesome parameter tuning process. In this project, from a different perspective, we propose to construct DLNs with sparse coding models as fundamental units. The sparse coding models that will be studied include single-layer and two-layer models, while the DLNs will be constructed based on single-layer models, two-layer models, as well as a combination of both. On one hand, we will establish the fundamental theory of the sparse coding models-based DLNs through theoretical analysis. On the other hand, we will devise a series of DLNs appropriate for real applications through extensive experiments. In view of the simple structure and excellent performance of the sparse coding models, this new integration method for sparse coding and deep learning is more natural and can help to circumvent the inheren

英文关键词： deep learning；sparse coding；neuroscience；neural network；recurrent connection