真实自发情感的听视觉多模态实时心理学连续维度分析

项目名称： 真实自发情感的听视觉多模态实时心理学连续维度分析

项目编号： No.61273265

项目类型： 面上项目

立项/批准年度： 2013

项目学科： 自动化技术、计算机技术

项目作者： 蒋冬梅

作者单位： 西北工业大学

项目金额： 79万元

中文摘要： 录制听视觉真实自发情感数据库，进行情感类别和心理学情感维度多层标注；提取语音情感特征、面部表情特征和语言关键词特征，基于心理学维度空间，采用转换卡尔曼滤波器(SKF)对情感状态进行实时连续估计，考察情感特征对状态估计的有效性;设计基于SKF的多模态动态贝叶斯网络(DBN)模型，实现听视觉融合的实时连续情感维度估计，提高情感分析的准确性和鲁棒性；建立状态耦合DBN模型，以估计到的心理学情感维度作为观测序列，利用DBN在线推理算法，对语音或视频中的情感进行即时分割和分类。项目克服了传统情感识别方法的两个缺点：1)只有在一段视频结束之后才能识别情感类别的缺点，能够实时进行情感状态分析；2)只能对有限类别情感进行离散划分的缺点，能得到情感的心理学连续维度变化，描述复杂的复合情感。该研究在智能监控、人机交互、自适应游戏设计、自闭症和老年痴呆症辅助治疗、服务质量评价等领域有重要研究意义和广泛应用前景。

中文关键词： 听视觉真实自发情感数据库；离散情感识别；连续情感维度估计；DBLSTM-RNN；DRNN

英文摘要： This project focuses on the on-line audio visual multi-modal analysis of spontaneous emotions in the psychological continuous dimensional space. Firstly an audio visual spontaneous emotion database will be recorded and annotated with emotion labels as well as the arousal and valence values. After the emotional features are extracted from speech, face image sequence and key words, a switching Kalman filter is designed to estimate the arousal and valence values of the emotion on-line. To improve the accuracy and robustness of the estimation, multi-modal Dynamic Bayesian Network (DBN) models based on switching Kalman filter will be designed to fuse the audio visual and linguistic features. Finally a coupled DBN model will be built to classify and segment the emotions in a video, with the estimated arousal and valence values as input. The advantages of the proposed methods lies in: 1) on-line affect state analysis can be done on speech or face videos, whereas the traditional emotion recognition methods only can get the results after the whole video ends; 2) continuous arousal and valence dimensions can be obtained to depict the complex non-basic emotion patterns, whereas the traditional methods only classify emotions with limited labels and can't depict the composite emotions in real life. This research is essential

英文关键词： audio visual spontaneous emotion database；categorical emotion recognition；continuous affect recognition；DBLSTM-RNN；DRNN