基于超声图像的静音语音识别关键技术研究

项目名称： 基于超声图像的静音语音识别关键技术研究

项目编号： No.61304250

项目类型： 青年科学基金项目

立项/批准年度： 2014

项目学科： 自动化技术、计算机技术

项目作者： 路文焕

作者单位： 天津大学

项目金额： 26万元

中文摘要： 基于发音器官的运动来识别语音（静音语音识别）可应用到广泛的领域，如识别喉切除的病人说话;在会场及电影院等需要保持静音或者隐私的场所不出声音只运动发音器官来打电话;再比如在高噪声环境中进行语音识别等。本课题将用超声仪结合摄像头来采集说话人的发音器官运动，基于唇部运动图像及舌头运动图像来获取发音器官运动,从而识别相应的语音。本课题将基于实现静音语音识别中的几个关键技术点来展开研究。首先，将利用有限波尔兹曼机通过非监督学习对摄像机记录的唇部运动图像及用超声仪记录的舌头运动图像分别进行特征提取及非线性的降维。为了实现多模态发音运动数据的融合，本课题利用多个有限波尔兹曼机搭建一个深度神经网络来对多通道、多模态的发音运动数据进行数据融合研究。基于多模态融合获得的发音运动数据特征，利用隐马尔科夫模型进行静音语音识别研究。本课题将在发音运动特征提取、多通道数据融合、发音运动识别等方面有创新性成果。

中文关键词： 超声图像；语音识别；语音生成；发音运动；深度学习

英文摘要： Speech recognition based on articulator movement (silent speech recognition)can be applied widely in many areas, such as speech recognition for patients with laryngectomy;making calls but speaking without sound in the situations like venue and cinemas where we need to keep silent or privacy,speech recognition in high-noise environments and so on. This project will use ultrasound machine and cameras to acquire images about speaker's articulator and recognize articulator's movement to get corresponding speech by the images. The research will focus on several key technical points about silent speech recognition. First, we will use constrained Boltzmann machine and unsupervised learning to extract features and decrease dimension nonlinearly for images about lips and tongues movements recorded by camera and ultrasonund machine,respectively. The research uses multiple constrained Boltzmann machines to build a deep neural network to fuse multi-channel and multi-modal data.We will use hidden Markov model in silent speech recognition by the characteristics derived from the fusion of multi-modal data.In the research, there will be innovative achievements in the feature extraction, multi-channel data fusion, articulator recognition and so on.

英文关键词： Ultrasound Image；Speech Recognition；Speech Production；Articulation；Deep learning