深层学习模型的复杂性:调查 (Model Complexity of Deep Learning: A Survey)

Model complexity is a fundamental problem in deep learning. In this paper we conduct a systematic overview of the latest studies on model complexity in deep learning. Model complexity of deep learning can be categorized into expressive capacity and effective model complexity. We review the existing studies on those two categories along four important factors, including model framework, model size, optimization process and data complexity. We also discuss the applications of deep learning model complexity including understanding model generalization capability, model optimization, and model selection and design. We conclude by proposing several interesting future directions.

翻译：模型复杂性是深层学习的一个根本问题。在本文件中,我们系统地概述了关于深层学习中模型复杂性的最新研究。深层学习的模型复杂性可以分为表现能力和有效模型复杂性。我们按照四个重要因素,包括示范框架、模型规模、优化过程和数据复杂性,审查关于这两类的现有研究。我们还讨论了深层学习模型复杂性的应用,包括理解模型的简单化能力、模型优化以及模型的选择和设计。我们最后提出了几个有趣的未来方向。