灵活可调适计算时间 (Layer Flexible Adaptive Computational Time)

Deep recurrent neural networks perform well on sequence data and are the model of choice. However, it is a daunting task to decide the structure of the networks, i.e. the number of layers, especially considering different computational needs of a sequence. We propose a layer flexible recurrent neural network with adaptive computation time, and expand it to a sequence to sequence model. Different from the adaptive computation time model, our model has a dynamic number of transmission states which vary by step and sequence. We evaluate the model on a financial data set and Wikipedia language modeling. Experimental results show the performance improvement of 7\% to 12\% and indicate the model's ability to dynamically change the number of layers along with the computational steps.

翻译：深层的经常性神经网络在序列数据方面表现良好,是选择的模式。然而,决定网络的结构,即层数,是一项艰巨的任务,特别是考虑到一个序列的不同计算需要。我们建议一个具有适应性计算时间的多层灵活的循环神经网络,并将其扩展为序列模型的顺序。不同于适应性计算时间模型,我们的模型有动态的传输状态,它们因步骤和顺序而异。我们评估财务数据集和维基百科语言模型的模型。实验结果显示7 ⁇ 至12 ⁇ 的性能改进,并显示模型在计算步骤的同时动态地改变层数的能力。