Biologically Plausible Learning using GAIT-prop Scales to ImageNet

Many of the recent advances in the field of artificial intelligence have been fueled by the highly successful backpropagation of error (BP) algorithm, which efficiently solves the credit assignment problem in artificial neural networks. However, it is unlikely that BP is implemented in its usual form within biological neural networks, because of its reliance on non-local information in propagating error gradients. Since biological neural networks are capable of highly efficient learning and responses from BP trained models can be related to neural responses, it seems reasonable that a biologically viable approximation of BP underlies synaptic plasticity in the brain. Gradient Adjusted Incremental Target Propagation (GAIT-prop) has recently been derived directly from BP and has been shown to successfully train networks in a more biologically plausible manner. However, so far, GAIT-prop has only been shown to work on relatively low-dimensional problems, such as handwritten-digit recognition. This work addresses some of the scaling issues in GAIT-prop and shows it to achieve performance comparable to BP on the much more challenging CIFAR10 and ImageNet datasets.