Invertible neural networks (INNs) are neural network architectures with invertibility by design. Thanks to their invertibility and the tractability of Jacobian, INNs have various machine learning applications such as probabilistic modeling, generative modeling, and representation learning. However, their attractive properties often come at the cost of restricting the layer designs, which poses a question on their representation power: can we use these models to approximate sufficiently diverse functions? To answer this question, we have developed a general theoretical framework to investigate the representation power of INNs, building on a structure theorem of differential geometry. The framework simplifies the approximation problem of diffeomorphisms, which enables us to show the universal approximation properties of INNs. We apply the framework to two representative classes of INNs, namely Coupling-Flow-based INNs (CF-INNs) and Neural Ordinary Differential Equations (NODEs), and elucidate their high representation power despite the restrictions on their architectures.
翻译:不可逆的神经网络(INNs)是心血管网络结构,其设计是不可视的。由于这些网络的可视性和Jacobian的可移动性,INNs拥有各种机器学习应用,如概率模型、基因模型和代表性学习。然而,它们的吸引力往往以限制层图设计为代价,这对其代表力提出了问题:我们能否使用这些模型来估计充分多样的功能?为了回答这个问题,我们制定了一个一般性理论框架来调查INNs的代表性,以差异几何结构的理论为根据。框架简化了二变形学的近似问题,使我们能够展示INS的普遍近似特性。我们把框架应用到INS的两个有代表性的类别,即Coubling-Flow IMNs(CF-INNs)和Neural Commondal Equations(NODs),尽管对其结构有限制,但我们也阐明了他们的高代表性。