Anderson-type acceleration method for Deep Neural Network optimization

In this paper we consider the neural network optimization for DNN. We develop Anderson-type acceleration method for the stochastic gradient decent method and it improves the network permanence very much. We demonstrate the applicability of the method for DNN and CNN. We discuss the application of the general class of the neural network design for computer tomography and inverse medium problems.