Although Deep Learning (DL) has achieved success in complex Artificial Intelligence (AI) tasks, it suffers from various notorious problems (e.g., feature redundancy, and vanishing or exploding gradients), since updating parameters in Euclidean space cannot fully exploit the geometric structure of the solution space. As a promising alternative solution, Riemannian-based DL uses geometric optimization to update parameters on Riemannian manifolds and can leverage the underlying geometric information. Accordingly, this article presents a comprehensive survey of applying geometric optimization in DL. At first, this article introduces the basic procedure of the geometric optimization, including various geometric optimizers and some concepts of Riemannian manifold. Subsequently, this article investigates the application of geometric optimization in different DL networks in various AI tasks, e.g., convolution neural network, recurrent neural network, transfer learning, and optimal transport. Additionally, typical public toolboxes that implement optimization on manifold are also discussed. Finally, this article makes a performance comparison between different deep geometric optimization methods under image recognition scenarios.
翻译:虽然深入学习(DL)在复杂的人工智能(AI)任务中取得了成功,但是它遇到了各种臭名昭著的问题(例如,特征冗余,以及消失或爆炸梯度),因为欧几里德空间的更新参数无法充分利用解决方案空间的几何结构。作为一种有希望的替代解决方案,以里曼尼亚为基础的DL利用几何优化来更新里曼尼方块的参数,并能够利用基本的几何信息。因此,本文章对在DL应用几何优化的情况进行了全面调查。首先,本文章介绍了几何优化的基本程序,包括各种几何优化器和里曼多元的一些概念。随后,本文章调查了不同DL网络在各种AI任务中应用几何优化的情况,例如,卷发神经网络、经常性神经网络、转移学习和最佳运输。此外,也讨论了在图像识别情景下采用优化的典型公共工具箱。最后,本文章对不同深度几何优化方法的绩效作了比较。