基于复杂语义的个性化图像集摘要研究

项目名称： 基于复杂语义的个性化图像集摘要研究

项目编号： No.61502138

项目类型： 青年科学基金项目

立项/批准年度： 2016

项目学科： 计算机科学学科

项目作者： 赵烨

作者单位： 合肥工业大学

项目金额： 19万元

中文摘要： 随着互联网技术及社会媒体的蓬勃发展，网络图像数据量呈爆炸式增长。我们如何利用丰富的社会媒体资源生成满足用户个性化需求的图像集摘要成为迫切需求。然而，对于现有的基于多模态媒体信息的图像集摘要研究集中于通用的、简单语义概念的研究，这很难满足用户个性化的需求和视觉复杂语义的体现。本课题致力于研究网络图像集个性化摘要的关键技术，通过建立多模态信息体系来实现跨媒体信息的融合，利用层次化媒体数据语义理解方法建立实体-描述-语义的关联。将跨媒体数据进行深度分析并映射到高层复杂语义空间，在语义关联的基础上进行聚合。利用超图对多模态数据及其关系建模，通过密集子图发现模型实现代表图像查找。通过对用户个人信息建模，挖掘用户的兴趣点和偏好，提出满足用户需求的个性化摘要设计方案。本课题拟构建一个个性化图像集摘要生成系统，将有力推动社会媒体挖掘技术和图像分析与检索理论。

中文关键词： 图像集摘要；个性化；复杂语义；多模态信息融合

英文摘要： With the rapid development of Internet technology and social media, the amount of digital images on the network increases dramatically. How we can leverage the rich social media information, including different types of objects and relations among these objects, becomes an increasingly important problem. Conventional research on image collection summarization analysis usually focuses on the simple media information, and this makes the technology difficult to deal with users' complex requirements. In this proposal, we target to address the key problems in image collection summarization by leveraging the knowledge mined from image social communities. We propose a method to build a multi-source information fusion by Webpage visual structure analysis and natural language understanding technology. Sample is evaluated by image quality evaluation system. Thus, there is a priori bias when selecting samples. We model the various objects and high-order relations in social media communities by hypergraphs instead of traditional graphs. We propose a new method for representative image selection based on dense sub-hypergraph. Through the user's personal information modeling, we exploit user’s interests and design a scheme for personalized image collection summarization. This proposal will present a personalized system for image collection summarization, which we believe, will advance the development of not only social media mining application but also image analysis and retrieval theories.

英文关键词： Image collection summarization;personality;complex semantics;multimodality information fusion