互联网有限监督信息下的通用图像语义理解机制研究

项目名称： 互联网有限监督信息下的通用图像语义理解机制研究

项目编号： No.61472276

项目类型： 面上项目

立项/批准年度： 2015

项目学科： 其他

项目作者： 韩亚洪

作者单位： 天津大学

项目金额： 80万元

中文摘要： 社会媒体和移动互联应用的发展使得互联网图像数据海量涌现，社会属性和不断增长的规模给图像语义理解带来了巨大挑战。尽管用户评论和标签为图像语义分析提供了更多的语义线索，但这些语义标注信息往往是有噪音和弱标记的。因此，可用的监督信息相对于巨大的语义输出空间是非常有限的。同时，由于负例或测试图像样本在理论上存在的无限语义空间，我们无法有效获得其语义空间的先验信息。本课题拟建立互联网有限监督信息下的通用图像语义理解框架，在结合多媒体、计算机视觉、机器学习和自然语言处理等交叉领域的最新进展和热点研究的基础上，主要研究：语义层次监督信息下的大规模字典学习、半监督异构的跨域学习、图像结构化语义预测和语义描述生成、以及算法高效求解和一致性分析等内容。通过集成相关研究成果与技术，发布基于本体语义的图像检索系统，为图像语义理解在互联网媒体搜索、监管与服务等领域的实际应用提供技术支撑和平台示范。

中文关键词： 图像语义理解；有限监督；通用；字典学习；跨域学习

英文摘要： With the popularity of social media and mobile internet applications, there is an explosive growth of web images. The social characteristics and the increased scalability turn out a great challenge in the image semantic understanding. Though the user's comments and tagging can be well exploited to provide more semantic cues for image semantic analysis, the annotations of these data contain a lot of noisy tags and are always weakly tagging. Thus, the supervision information available is limited due to the huge output space. Furthermore, the negative or testing examples come from an infinite semantic space and we have no clue about the semantic these examples include. In this proposal, we target to develop a framework of ad hoc web image semantic understanding with limited supervision. Based on the recent development and research focuses in multimedia, computer vision, machine learning, and natural language processing, four key issues are explored in this proposal: large-scale supervised dictionary learning with semantic taxonomy, semi-supervised heterogeneous domain adaptation, structural prediction and description generation of image semantic, efficient algorithm and its consistency analysis. Based on the research results of this proposal and related technologies, we will release a portal of image retrieval with semantic ontology. The output technologies and demonstration of this proposal will mainly contribute to the real-world applications of image semantic understanding in web multimedia search, regulation, and services etc.

英文关键词： Image Semantic Understanding;Limited Supervision;Ad Hoc;Dictionary Learning;Domain Adaptation