In the past few decades, there has been an explosion in the amount of available data produced from various sources with different topics. The availability of this enormous data necessitates us to adopt effective computational tools to explore the data. This leads to an intense growing interest in the research community to develop computational methods focused on processing this text data. A line of study focused on condensing the text so that we are able to get a higher level of understanding in a shorter time. The two important tasks to do this are keyword extraction and text summarization. In keyword extraction, we are interested in finding the key important words from a text. This makes us familiar with the general topic of a text. In text summarization, we are interested in producing a short-length text which includes important information about the document. The TextRank algorithm, an unsupervised learning method that is an extension of the PageRank (algorithm which is the base algorithm of Google search engine for searching pages and ranking them) has shown its efficacy in large-scale text mining, especially for text summarization and keyword extraction. this algorithm can automatically extract the important parts of a text (keywords or sentences) and declare them as the result. However, this algorithm neglects the semantic similarity between the different parts. In this work, we improved the results of the TextRank algorithm by incorporating the semantic similarity between parts of the text. Aside from keyword extraction and text summarization, we develop a topic clustering algorithm based on our framework which can be used individually or as a part of generating the summary to overcome coverage problems.
翻译:在过去几十年中,从不同主题的不同来源产生的可用数据数量急剧增加。 如此庞大的数据的提供使得我们不得不采用有效的计算工具来探索数据。 这导致研究界对开发侧重于处理文本数据的计算方法的兴趣日益浓厚, 以开发侧重于处理文本数据的计算方法。 一行研究的重点是压缩文本, 以便我们能够在更短的时间内获得更高程度的理解。 这样做的两大任务是关键词提取和文本摘要化。 在关键词提取中, 我们有兴趣从文本中找到关键的重要词。 这使我们熟悉文本的一般主题。 在文本摘要化中, 我们感兴趣的是生成一个包含文件重要信息的短长的文本。 TextRank 算法, 是一个非超强的学习方法, 是 PageRank (algorthm) 的扩展。 这是谷歌搜索页面和排序的基算法 。 在大规模文本摘要挖掘中, 特别是文本摘要化和关键词提取中, 我们的算法可以自动地解析出文本中的重要部分, 也就是我们使用不同语言序列的结果 。