基于深度学习的装配场景理解及装配诱导、监测研究

项目名称： 基于深度学习的装配场景理解及装配诱导、监测研究

项目编号： No.51475251

项目类型： 面上项目

立项/批准年度： 2015

项目学科： 机械、仪表工业

项目作者： 陈成军

作者单位： 青岛理工大学

项目金额： 80万元

中文摘要： 装配工艺诱导及监测在机械设备装配、维修、培训和车间级装配质量控制等领域具有广泛的应用前景，实现由人、真实零部件和数字化装配模型构成装配空间的融合、交互和理解是实现诱导、监测的关键。本研究提出基于深度学习的装配场景理解的概念，通过装配场景理解，识别装配任务、生成诱导信息、监测装配操作工艺，其核心科学问题是装配场景中各类特征的分层学习和抽象，重点研究内容包括：①模拟人类视觉感知机制，建立基于深度学习的3D零件识别框架；②建立基于手部RGB-D图像序列深度学习的装配动作识别模型，揭示手部动作与装配操作间的映射关系；③建立零件-装配操作驱动的产品装配工艺粒度模型，实现装配工艺智能诱导和监测；④建立基于头部RGB-D图像稀疏编码的增强现实注册模型，揭示头部方位与其RGB-D图像特征间的关系。研究成果不仅可为装配诱导、装配质量控制、装配人机功效学、计算机交互等研究提供理论参考，还可推广应用。

中文关键词： 装配诱导与监测；RGB-D图像；深度学习；增强现实；装配场景理解

英文摘要： Guiding and monitoring of assembly process have wide application foregrounds in areas such as assembly, maintenance, training and workshop-level assembly quality control of complicated mechanical equipments. The key technologies to realize the guiding and monitoring of assembly process is to make overlaying, interacting and understanding among operator, physical componencts and digital assembly model. This research proposes a concept named deep learning-based assembly scene understanding. With the deep learning theory, the assembly scene can be understood. Subsequently the assembly task is recongnized, then the guiding and monitoring of assembly process is realized. The main scientific problems are the hierarchical learning and extractions of features in the assembly scene. Based on this point, the research contents are follows. Firstly， a deep learning framework to recognize 3D mechanical parts is established by simulating human visual perception mechanism. Secondly, an assembly action recognition method is modeled to reveal the mapping of hand movement and assembly operations by deep learning of sequtial RGB-D hand images of hands. Thirdly the parts-assembly action driven assembly process guiding and monitoring framework is built. Lastly an augmented reality registration method using the sparse coding of head RGB-D images is presented to model the relationship between head gesture and features of head RGB-D images. Results in this research will not only provide atheoretical reference for research like assembly guiding, assembly quality control, assembly ergonomics and human computer interaction, but also spread into related applications.

英文关键词： Assembly guiding and monitoring;RGB-D Images;Deep Learning;Augmented reality;Assembly Scene Understanding