Complex-valued neural networks (CVNNs) have been widely applied to various fields, especially signal processing and image recognition. However, few works focus on the generalization of CVNNs, albeit it is vital to ensure the performance of CVNNs on unseen data. This paper is the first work that proves a generalization bound for the complex-valued neural network. The bound scales with the spectral complexity, the dominant factor of which is the spectral norm product of weight matrices. Further, our work provides a generalization bound for CVNNs when training data is sequential, which is also affected by the spectral complexity. Theoretically, these bounds are derived via Maurey Sparsification Lemma and Dudley Entropy Integral. Empirically, we conduct experiments by training complex-valued convolutional neural networks on different datasets: MNIST, FashionMNIST, CIFAR-10, CIFAR-100, Tiny ImageNet, and IMDB. Spearman's rank-order correlation coefficients and the corresponding p values on these datasets give strong proof that the spectral complexity of the network, measured by the weight matrices spectral norm product, has a statistically significant correlation with the generalization ability.
翻译:复杂的神经网络(CVNNs)被广泛应用于各个领域,特别是信号处理和图像识别,然而,很少有工作侧重于CVNNs的普遍化,尽管确保CVNNs在不可见数据方面的性能至关重要。本文是首次证明对复杂、有价值的神经网络进行总体化的首份工作。光谱复杂度的结合尺度是重量矩阵的光谱标准产品。此外,我们的工作为CVNs提供了在培训数据是连续的,也受光谱复杂程度影响,培训数据是相继的,也受到光谱复杂程度的影响时,CVNNSNs提供了通用的界限。理论上,这些界限是通过Maurey Sparsificization Lemmma 和 Dudley Entropy Includicly 生成的,尽管这对确保CVNNNNNN的功能至关重要。这是首次证明CVNNE网络具有普遍价值的通用神经神经网络(MNIST、FAshion MINST、FIS 网络、CIF COM 能力测得的系统 、测得的光谱质的系统测量等的光能、测重等的光能的基能、测成的基能的基重的基的基的基的基的基的基的基的基的基的基的基的基能的基的基能的基能、测量的基的基的基的基的基的基的基的基的基的基的基的基的基的基的基的基的基的基的基的基的基的基的基的基的基的基的基的基的基的基的基的基的基的基的基的基的基的基的基的基的基的基的基的基的基的基的基的基的基的基的基的基的基的基的基的基的基的比。