面向多模态信息有效利用的机器学习方法研究

项目名称： 面向多模态信息有效利用的机器学习方法研究

项目编号： No.61273301

项目类型： 面上项目

立项/批准年度： 2013

项目学科： 自动化技术、计算机技术

项目作者： 姜远

作者单位： 南京大学

项目金额： 80万元

中文摘要： 随着大量多模态数据的收集，对多模态数据进行智能处理成为迫切需要。为了有效地利用多模态数据提供的信息，本项目对机器学习利用多模态信息所面临的四个问题进行研究。首先，不同模态所包含的信息存在强弱之分，并且不同模态的特征抽取代价不同，如何利用弱模态信息进行快速、有效的多模态学习？第二，很多情况下，多模态间的信息并非依据类别条件独立，如何在样本极少的半监督环境下对多模态特征进行选择和抽取，尽量使得模态之间的条件独立性得以满足？第三，多模态信息可能因为各种原因而缺失，例如传感设备故障导致模态信息缺失，如何利用不全面的模态信息进行学习？第四，当多模态信息并没有显式地划分为多个模态，而是隐藏在单模态特征中，如何对隐多模态信息加以划分和利用，并使用协同训练类方法加以学习？本项目将为上述问题提供解决方案并研制原型系统，发表国际期刊/会议和国内一级学报论文8-10篇，申请专利2-3项，培养3-5名研究生。

中文关键词： 多模态信息；机器学习；多视图学习；；

英文摘要： As a large number of multimodal data collected, it becomes desirable to process these data intelligently. To effectively utilize the information contained in multimodal data, we focus on four issues on machine learning techniques while making use of multimodal information. 1. The information from different modals is not with equal importance, i.e., some modals are with rich information while others are poor, and the feature extraction costs are also different for different modals. How to effectively exploit the information form poor modals for fast multimodal learning? 2. Information in different modals is not conditional independent in most case. In order to make modals conditionally independ to each other, how to select or extract features for multimodal data when examples are limited in semi-supervised scenario? 3. Multimodal information can be incomplete for many reasons, e.g., failures of sensors, how to learning with these incomplete multimodal information? 4. How to learning with hidden multimodal data, where the multimodal information is not divided into multiple modals, but hided in a single modal representation, by co-training style approaches? As an application of the solutions to these four issues, we will also build a prototype system. It is expected to publish 8-10 high quality papers on important

英文关键词： Multi-Modal Information；Machine Learning；Multi-View Learning；；