长短期记忆(LSTM)是一种用于深度学习领域的人工递归神经网络(RNN)架构。与标准前馈神经网络不同,LSTM具有反馈连接。它不仅可以处理单个数据点(例如图像),而且可以处理整个数据序列(例如语音或视频)。例如,LSTM适用于诸如未分段的连接手写识别,语音识别和网络流量或IDS(入侵检测系统)中的异常检测之类的任务。常见的LSTM单元由单元,输入门,输出门和忘记门组成。单元会记住任意时间间隔内的值,并且三个门控制着进出单元的信息流。LSTM网络非常适合基于时间序列数据进行分类,处理和做出预测,因为时间序列中重要事件之间可能存在未知持续时间的滞后。开发LSTM是为了解决训练传统RNN时可能遇到的梯度消失问题。与缝隙长度相对不敏感是LSTM在众多应用中优于RNN,隐马尔可夫模型和其他序列学习方法的优势。

最新论文

Recurrent neural networks are important tools for sequential data processing. However, they are notorious for problems regarding their training. Challenges include capturing complex relations between consecutive states and stability and efficiency of training. In this paper, we introduce a recurrent neural architecture called Deep Memory Update (DMU). It is based on updating the previous memory state with a deep transformation of the lagged state and the network input. The architecture is able to learn to transform its internal state using any nonlinear function. Its training is stable and fast due to relating its learning rate to the size of the module. Even though DMU is based on standard components, experimental results presented here confirm that it can compete with and often outperform state-of-the-art architectures such as Long Short-Term Memory, Gated Recurrent Units, and Recurrent Highway Networks.

0
0
下载
预览
参考链接
Top
微信扫码咨询专知VIP会员