Attention机制最早是在视觉图像领域提出来的，但是真正火起来应该算是google mind团队的这篇论文《Recurrent Models of Visual Attention》[14]，他们在RNN模型上使用了attention机制来进行图像分类。随后，Bahdanau等人在论文《Neural Machine Translation by Jointly Learning to Align and Translate》 [1]中，使用类似attention的机制在机器翻译任务上将翻译和对齐同时进行，他们的工作算是是第一个提出attention机制应用到NLP领域中。接着类似的基于attention机制的RNN模型扩展开始应用到各种NLP任务中。最近，如何在CNN中使用attention机制也成为了大家的研究热点。下图表示了attention研究进展的大概趋势。
VIP内容
摘要：随着深度学习技术的快速发展，许多研究者尝试利用深度学习来解决文本分类问题，特别是在卷积神经网络和循环神经网络方面，出现了许多新颖且有效的分类方法。对基于深度神经网络的文本分类问题进行分析，介绍卷积神经网络、循环神经网络、注意力机制等方法在文本分类中的应用和发展，分析多种典型分类方法的特点和性能，从准确率和运行时间方面对基础网络结构进行比较，表明深度神经网络较传统机器学习方法在用于文本分类时更具优势，其中卷积神经网络具有优秀的分类性能和泛化能力。在此基础上，指出当前深度文本分类模型存在的不足，并对未来的研究方向进行展望。
http://www.ecice06.com/article/2021/1000-3428/2121.htm
文本分类技术经历了从专家系统到机器学习再到深度学习的发展过程。在20世纪80年代以前，基于规则系统的文本分类方法需要领域专家定义一系列分类规则，通过规则匹配判断文本类别。基于规则的分类方法容易理解，但该方法依赖专家知识，系统构建成本高且可移植性差。20世纪90年代，机器学习技术逐渐走向成熟，出现了许多经典的文本分类算法，如决策树[1]、朴素贝叶斯[2]、支持向量机[3]、最大熵[4]、最近邻[5]等，这些方法部分克服了上述缺点，一定程度上实现了分类器的自动生成，被广泛应用于各个领域。然而，机器学习方法在构建分类器之前通常需要繁杂的人工特征工程，这限制了其进一步发展。
2012年之后，深度学习算法引起了研究者的广泛关注。深度学习为机器学习建模提供了一种直接端到端的解决方案，可避免复杂的特征工程。GolVe[6]和word2vec[7]等词向量模型的提出，使深度学习算法成功地应用到文本处理领域，随后出现了各种基于深度神经网络（Deep Neural Network，DNN）的文本分类方法。这些方法主要采用卷积神经网络（Convolutional Neural Network，CNN）、循环神经网络（Recurrent Neural Network，RNN）和注意力机制等深度学习技术，并且取得了比传统方法更为出色的性能。近年来，图卷积网络（Graph Convolutional Network，GCN）、区域嵌入和元学习等一些新的深度学习方法也被应用于文本分类领域。
本文对基于深度神经网络的文本分类技术进行介绍和分析，阐述卷积神经网络、循环神经网络和注意力机制等方法在文本分类中的应用和发展情况，总结各类方法的特点及区别，并对不同方法的性能表现和适用场景进行比较，讨论在应用深度学习方法处理文本分类任务时应当注意的问题。在此基础上，指出针对该技术未来的研究方向。
成为VIP会员查看完整内容
基于深度学习的文本分类技术研究进展
热门内容
最新内容
Increasing numbers of patients with disabilities or elderly people with mobility issues often suffer from a pressure ulcer. The affected areas need regular checks, but they have a difficulty in accessing a hospital. Some remote diagnosis systems are being used for them, but there are limitations in checking a patient's status regularly. In this paper, we present a remote medical assistant that can help pressure ulcer management with image processing techniques. The proposed system includes a mobile application with a deep learning model for wound segmentation and analysis. As there are not enough data to train the deep learning model, we make use of a pretrained model from a relevant domain and data augmentation that is appropriate for this task. First of all, an image preprocessing method using bilinear interpolation is used to resize images and normalize the images. Second, for data augmentation, we use rotation, reflection, and a watershed algorithm. Third, we use a pretrained deep learning model generated from skin wound images similar to pressure ulcer images. Finally, we added an attention module that can provide hints on the pressure ulcer image features. The resulting model provides an accuracy of 99.0%, an intersection over union (IoU) of 99.99%, and a dice similarity coefficient (DSC) of 93.4% for pressure ulcer segmentation, which is better than existing results.
最新论文
Increasing numbers of patients with disabilities or elderly people with mobility issues often suffer from a pressure ulcer. The affected areas need regular checks, but they have a difficulty in accessing a hospital. Some remote diagnosis systems are being used for them, but there are limitations in checking a patient's status regularly. In this paper, we present a remote medical assistant that can help pressure ulcer management with image processing techniques. The proposed system includes a mobile application with a deep learning model for wound segmentation and analysis. As there are not enough data to train the deep learning model, we make use of a pretrained model from a relevant domain and data augmentation that is appropriate for this task. First of all, an image preprocessing method using bilinear interpolation is used to resize images and normalize the images. Second, for data augmentation, we use rotation, reflection, and a watershed algorithm. Third, we use a pretrained deep learning model generated from skin wound images similar to pressure ulcer images. Finally, we added an attention module that can provide hints on the pressure ulcer image features. The resulting model provides an accuracy of 99.0%, an intersection over union (IoU) of 99.99%, and a dice similarity coefficient (DSC) of 93.4% for pressure ulcer segmentation, which is better than existing results.
