大规模数据集3D手语识别的研究

项目名称： 大规模数据集3D手语识别的研究

项目编号： No.61472398

项目类型： 面上项目

立项/批准年度： 2015

项目学科： 计算机科学学科

项目作者： 柴秀娟

作者单位： 中国科学院计算技术研究所

项目金额： 80万元

中文摘要： 随着计算技术和设备的发展，体感交互已经成为新一代人机交互技术的代表。作为一种典型的3D体势动作，手语识别的研究具有非常重要的学术价值和社会意义。本项目选择非接触深度传感器和可视摄像机作为信息输入源，以手语识别问题作为研究对象。面向大词汇量3D手语识别问题，从手语的构成要素出发，研究融合关键手型、运动轨迹以及面部表情的多通道3D手语特征表示，并通过哈希编码，流形划分以及格拉斯曼流形投影分别获取手形、轨迹和表情各维度特征的低维表示，用于后续的识别。在手语训练数据不足的情况下，本项目拟将源域与目标域共性特征作为桥梁，将知识有效迁移至数据量较少的目标域，实现非特定人自适应的手语建模。在面向应用需求的高效连续手语识别上，考虑不同词语之间跳转和畸变的影响，将HMM与N-Gram统计语法模型结合，以得到快速准确的句子识别结果。这一研究同时有望能够实现低成本手语自动理解的目标。

中文关键词： 体感交互；手语识别；手语特征表示；非特定人手语建模；连续手语识别

英文摘要： With the development of computing technologies and devices, the human body interaction has become one of the typical technologies in the new generation human computer interaction. As a typical 3D body activity, the study on sign language recognition (SLR) has very important academic value and social significance. In this project, we propose to use a depth camera and visual camera as input sources, and target the problem of automatic recognition of sign language. For the large vocabulary sign language recognition, considering on the main elements of sign language, this project will explore the multimodal 3D sign language representation by fusing key hand posture, motion trajectory and facial expression. The low-dimensional features will be obtained through Hash coding, manifold partition and Grassmannian manifold projection techniques. With limited training data, in order to realize the signer independent sign language recognition, this project will take the common feature as the bridge and transfer the knowledge from source domain to target domain having less data. For the continuous sign language recognition with the real-time application requirements, the transition and distortion between signs are the biggest difficulties. This project will explore to combine the HMM and the N-Gram statistical model to get the accurate sentence recognition result efficiently. As a result, this project is expected to reach the goal of low cost sign language recognition system.

英文关键词： Body Sensing based Interaction;Sign Language Recognition;Feature Representation of Sign Language;Signer Independent Sign Language Modeling;Continuous Sign Language Recognition