In addition to being extremely non-linear, modern problems require millions if not billions of parameters to solve or at least to get a good approximation of the solution, and neural networks are known to assimilate that complexity by deepening and widening their topology in order to increase the level of non-linearity needed for a better approximation. However, compact topologies are always preferred to deeper ones as they offer the advantage of using less computational units and less parameters. This compacity comes at the price of reduced non-linearity and thus, of limited solution search space. We propose the 1-Dimensional Polynomial Neural Network (1DPNN) model that uses automatic polynomial kernel estimation for 1-Dimensional Convolutional Neural Networks (1DCNNs) and that introduces a high degree of non-linearity from the first layer which can compensate the need for deep and/or wide topologies. We show that this non-linearity enables the model to yield better results with less computational and spatial complexity than a regular 1DCNN on various classification and regression problems related to audio signals, even though it introduces more computational and spatial complexity on a neuronal level. The experiments were conducted on three publicly available datasets and demonstrate that, on the problems that were tackled, the proposed model can extract more relevant information from the data than a 1DCNN in less time and with less memory.
翻译:除了极端非线性之外,现代问题还需要数以百万计(如果不是数十亿)的参数才能解决,或者至少能够很好地接近解决方案,神经网络通过深化和扩大其表层来消化这一复杂性,从而提高更佳的近层所需的非线性水平;但是,由于使用计算单位较少和参数较少的优势,压缩的表层总是偏向于更深的表层;这种不线性的代价是非线性减少,因而是有限的解决方案搜索空间。我们提议采用1D1多级多线性多线性神经网络(DPNN)模型,该模型对1DVS Neural Neural网络(DCNN)采用自动多线性内核内核估计法,以便增加其非线性水平,从而弥补对深度和(或)宽度表性表的需要。我们表明,这种非线性使模型能够产生比正常的1DCNNNNN网络(DPNNNNNN)更好的结果。 尽管在1级的音频信号方面采用自动多的多的多圆内内内内内核估计,但是在1级的实验中,在1级和空间数据中可以较难的模型上显示,在1号中比较容易处理的神经的模型上进行较难。