Deep Reservoir Computing with Learned Hidden Reservoir Weights using Direct Feedback Alignment

Deep Reservoir Computing has emerged as a new paradigm for deep learning, which is based around the reservoir computing principle of maintaining random pools of neurons. The reservoir paradigm reflects and respects the high degree of recurrence in biological brains, and the role that neuronal dynamics play in learning. However, one issue hampering deep reservoir development is that one cannot backpropagate through the reservoir layers. Recent deep reservoir architectures do not learn hidden or hierarchical representations in the same manner as deep artifical neural neteworks (ANNs), but rather concatenate all hidden reservoirs together to perform traditional regression. Here we present a novel Deep Reservoir Computer for time series prediction and classification that learns through the non-differentiable hidden reservoir layers using a biologically-inspired backpropagation alternative called Direct Feedback Alignment, which resembles global dopamine signal broadcasting in the brain. The hope is that this will enable future deep reservoir architectures to learn hidden temporal representations.