SPINN: Sparse, Physics-based, and Interpretable Neural Networks for PDEs

We introduce a class of Sparse, Physics-based, and Interpretable Neural Networks (SPINN) for solving ordinary and partial differential equations. By reinterpreting a traditional meshless representation of solutions of PDEs we develop a class of sparse neural network architectures that are interpretable. The SPINN model we propose here serves as a seamless bridge between two extreme modeling tools for PDEs, dense neural network based methods and traditional mesh-free numerical methods, thereby providing a novel means to develop a new class of hybrid algorithms that build on the best of both these viewpoints. A unique feature of the SPINN model that distinguishes it from other neural network based approximations proposed earlier is that it is (i) fully interpretable, and (ii) sparse in the sense that it has much fewer connections than a dense neural network of the same size. Further, the SPINN algorithm implicitly encodes mesh adaptivity and is able to handle discontinuities in the solutions too. In addition we demonstrate that Fourier series representations can be expressed as a special class of SPINN and propose generalized neural network analogues of Fourier representations. We illustrate the utility of the proposed method with a variety of examples involving ordinary differential equations, elliptic, parabolic, hyperbolic and nonlinear partial differential equations, and an example in fluid dynamics.