We introduce Active Tuning, a novel paradigm for optimizing the internal dynamics of recurrent neural networks (RNNs) on the fly. In contrast to the conventional sequence-to-sequence mapping scheme, Active Tuning decouples the RNN's recurrent neural activities from the input stream, using the unfolding temporal gradient signal to tune the internal dynamics into the data stream. As a consequence, the model output depends only on its internal hidden dynamics and the closed-loop feedback of its own predictions; its hidden state is continuously adapted by means of the temporal gradient resulting from backpropagating the discrepancy between the signal observations and the model outputs through time. In this way, Active Tuning infers the signal actively but indirectly based on the originally learned temporal patterns, fitting the most plausible hidden state sequence into the observations. We demonstrate the effectiveness of Active Tuning on several time series prediction benchmarks, including multiple super-imposed sine waves, a chaotic double pendulum, and spatiotemporal wave dynamics. Active Tuning consistently improves the robustness, accuracy, and generalization abilities of all evaluated models. Moreover, networks trained for signal prediction and denoising can be successfully applied to a much larger range of noise conditions with the help of Active Tuning. Thus, given a capable time series predictor, Active Tuning enhances its online signal filtering, denoising, and reconstruction abilities without the need for additional training.
翻译:我们引入了“主动图示”,这是优化飞行中反复神经网络内部动态的新模式。与常规的顺序到序列映射计划相比,“主动图示”将RNN的经常性神经活动从输入流中分离出来,使用正在运行的时间梯度信号将内部动态调节到数据流中。因此,模型输出仅取决于其内部隐藏动态和自身预测的闭环反馈;其隐藏状态通过回溯分析信号观测与模型输出在时间上的差异所产生的时间梯度不断调整。与此不同的是,“主动图示”将信号积极但间接地从输入流中分离出来,将最可信的隐藏状态序列安装到观测中。我们展示了“主动图示”在数个时间序列预测基准上的有效性,包括多个超额正弦波、一个混乱的双倍双倍双倍平流和波波波波动态。积极的图案通过不断提高所有经评估模型的坚固性、准确性和总体化能力。此外,为信号预测和动态预测的动态预测能力而培训网络可以成功应用一个更大的在线预测和动态预测能力。