Learned Iterative Shrinkage-Thresholding Algorithm (LISTA) introduces the concept of unrolling an iterative algorithm and training it like a neural network. It has had great success on sparse recovery. In this paper, we show that adding momentum to intermediate variables in the LISTA network achieves a better convergence rate and, in particular, the network with instance-optimal parameters is superlinearly convergent. Moreover, our new theoretical results lead to a practical approach of automatically and adaptively calculating the parameters of a LISTA network layer based on its previous layers. Perhaps most surprisingly, such an adaptive-parameter procedure reduces the training of LISTA to tuning only three hyperparameters from data: a new record set in the context of the recent advances on trimming down LISTA complexity. We call this new ultra-light weight network HyperLISTA. Compared to state-of-the-art LISTA models, HyperLISTA achieves almost the same performance on seen data distributions and performs better when tested on unseen distributions (specifically, those with different sparsity levels and nonzero magnitudes). Code is available: https://github.com/VITA-Group/HyperLISTA.
翻译:在本文中,我们表明,在ListA网络中增加中间变量的动力可以实现更好的趋同率,特别是,具有最优实例参数的网络具有超线性趋同性。此外,我们的新理论结果导致一个基于前层自动和适应性地计算ListA网络层参数的实用方法。也许最令人惊讶的是,这种适应性参数程序会减少对ListA的培训,使其只能从数据中调整3个超参数:在ListA网络中最近改进了ListA复杂度后的新记录。我们称这个新的超光重网络超光速超光速/超光速参数网络为超光速超光速。与最先进的ListA模型相比,超光速LISTA在以前层为基础自动和适应性地计算ListA网络层的参数,并在测试无形分布时进行更好的表现(具体而言,那些具有不同孔径水平和非星级的LA-TAS Group)。