Recent advances in deep learning have resulted in image compression algorithms that outperform JPEG and JPEG 2000 on the standard Kodak benchmark. However, they are slow to train (due to backprop-through-time) and, to the best of our knowledge, have not been systematically evaluated on a large variety of datasets. In this paper, we perform the first large-scale comparison of recent state-of-the-art hybrid neural compression algorithms, while exploring the effects of alternative training strategies (when applicable). The hybrid recurrent neural decoder is a former state-of-the-art model (recently overtaken by a Google model) that can be trained using backprop-through-time (BPTT) or with alternative algorithms like sparse attentive backtracking (SAB), unbiased online recurrent optimization (UORO), and real-time recurrent learning (RTRL). We compare these training alternatives along with the Google models (GOOG and E2E) on 6 benchmark datasets. Surprisingly, we found that the model trained with SAB performs better (outperforming even BPTT), resulting in faster convergence and a better peak signal-to-noise ratio.
翻译:最近深层次学习的进展导致了图像压缩算法,比JPEG和JPEG 2000在标准Kodak基准上的图像压缩算法高得多,然而,这些算法在培训(由于反正通时间)方面速度缓慢,而且据我们所知,对大量数据集没有进行系统的评估。在本文中,我们首次对最新最先进的混合神经压缩算法进行了大规模比较,同时探索了替代培训战略(适用时)的效果。混合经常性神经解密器是以前最先进的模型(最近被谷歌模型取代 ), 可以使用后向通时间(BPTTT)或使用其他算法进行训练,如零星的注意力回溯跟踪(SAB)、公正的在线经常性优化(UORO)和实时经常性学习(RTRL ) 。我们将这些培训替代方法与谷歌模型(GOG和E2E)在6个基准数据集上进行对比。令人惊讶的是,我们发现,经过SAB培训的模型表现得更好(甚至比BPTTT),从而导致更快的趋近和最高峰信号。