Integral equations (IEs) are functional equations defined through integral operators, where the unknown function is integrated over a possibly multidimensional space. Important applications of IEs have been found throughout theoretical and applied sciences, including in physics, chemistry, biology, and engineering; often in the form of inverse problems. IEs are especially useful since differential equations, e.g. ordinary differential equations (ODEs), and partial differential equations (PDEs) can be formulated in an integral version which is often more convenient to solve. Moreover, unlike ODEs and PDEs, IEs can model inherently non-local dynamical systems, such as ones with long distance spatiotemporal relations. While efficient algorithms exist for solving given IEs, no method exists that can learn an integral equation and its associated dynamics from data alone. In this article, we introduce Neural Integral Equations (NIE), a method that learns an unknown integral operator from data through a solver. We also introduce an attentional version of NIE, called Attentional Neural Integral Equations (ANIE), where the integral is replaced by self-attention, which improves scalability and provides interpretability. We show that learning dynamics via integral equations is faster than doing so via other continuous methods, such as Neural ODEs. Finally, we show that ANIE outperforms other methods on several benchmark tasks in ODE, PDE, and IE systems of synthetic and real-world data.
翻译:综合方程式(IEs)是通过整体操作者定义的功能方程式,其中未知的功能在可能的多维空间中融合在一起。在理论和应用科学(包括物理、化学、生物学和工程)中,已经发现IEs的重要应用;往往以反问题的形式出现。IEs特别有用,因为差异方程式,例如普通差异方程式(ODE),和部分差异方程式(PDEs),可以用一个通常更方便解决的综合方程式来制定。此外,与ODE和PDEs不同的是,IEs可以模拟内在的非本地动态系统,例如具有长距离波形关系的系统。虽然在理论和应用科学(包括物理、化学、生物学和工程学)中发现了高效的算法,但没有任何方法能够单独从数据中学习一个整体方程式及其相关动态。在本文章中,我们引入了一个通过一个解答器从数据中学习一个未知的组合操作者的方法。我们还引入了一个注意的版本,称为“注意神经整体整体方程式”(ANIE),在这个系统中,一个内部的内置的内置系统(ANIE),例如有长波波波波关系,通过自我解释,另一个的计算法则通过直路路路路路路,而能度,通过直径路路路路,而使其他的计算方法可以改进了其他的自我判法,通过直径等式,通过直路路路路路路路,从而改进。