基于特权信息和面部先验知识的表情类别和动作单元识别研究

项目名称： 基于特权信息和面部先验知识的表情类别和动作单元识别研究

项目编号： No.61473270

项目类型： 面上项目

立项/批准年度： 2015

项目学科： 自动化技术、计算机技术

项目作者： 王上飞

作者单位： 中国科学技术大学

项目金额： 80万元

中文摘要： 本项目提出以红外热像为特权信息的可见光表情识别研究；在此基础上，研究融合面部先验知识和面部图像数据的可见光表情类别和动作单元识别；并探讨多标签的表情类别分类方法。 本项目以面部红外热图作为特权信息，在表情识别模型的训练阶段，辅助可见图像，更好地建立可见图像特征到表情之间的映射关系；在表情识别模型的测试阶段，只需通过可见光图像识别用户表情，提出基于特权信息的可见光表情分析和识别研究，为提高可见光表情识别对光照的鲁棒性提出了一种新的、可行的解决途径。本项目以面部动作单元之间以及表情类别和动作单元之间的时空概率关系为先验知识，结合特征驱动的表情识别方法，提出融合面部先验知识和面部图像数据的表情识别，为推高表情识别算法的扩展性能提供新的思路，必将推动表情识别的实用化进程。本项目对各种表情类别的共生和互斥关系进行分析建模，提出多标签的表情类别识别，为表情识别研究提供了新的方法。

中文关键词： 特权信息；面部先验知识；表情识别；面部动作单元识别

英文摘要： In this project, we propose to address several issues related to facial action unit and expression recognition. First, we propose to recognize expression from visible images using thermal infrared images as privileged information. Second, we propose to combine facial prior knowledge with facial images to improve facial action unit and expression recognition. Finally, we propose a multi-expression recognition by exploiting dependencies among expressions. First, facial thermal images are used as privileged information. During the training of the expression recognition model, the privileged information is used to help select more effective visible image features and to help establish a better mapping relationship between the visible image features and its expression labels. During testing, using the learned mapping function, the expressions can be directly inferred from the visible images using selected visible features without privileged information. The proposed expression recognition method learning from privileged information provides a novel and potential solution to enhance the robustness of visible facial expression recognition to illuminant changes. Second, we combine the facial prior knowledge on action unit and expressions with image-driven recognition methods, and introduce a facial prior-based expression and action unit recognition. Since the generic knowledge on facial action and expression is independent on the imaging conditions, the proposed facial prior-based expression and action unit recognition may greatly advance the application process of expression and action unit recognition. Third, we analyze and capture the global concurrent and mutual exclusive dependences among expressions, and propose a multi-label expression classification method. It provides a new method for expression analysis and modeling.

英文关键词： privileged information;facial prior knoweldge;expression recognition;action unit recognition