Approximating Ordinary and Partial Differential Equations Using the Theory of Connections and Least Squares Support Vector Machines

Differential equations are used as numerical models to describe physical phenomena throughout the field of engineering and science, including heat and fluid flow, structural bending, and systems dynamics. Although there are many other techniques for finding approximate solutions to these equations, this paper looks to compare the application of the Theory of Connections (ToC) with one based on Least-Squares Support Vector Machines (LS-SVM). The ToC method uses a constrained expression (an expression that always satisfies the differential equation constraints), which transforms the process of solving a differential equation into solving an unconstrained optimization problem, which is ultimately solved via least-squares. In addition to individual analysis, the two methods are merged into a new methodology, called constrained SMVs (CSVM), by incorporating the LS-SVM method into the ToC framework to solve unconstrained problems. Numerical tests are conducted on four sample problems: one first order linear ODE, one first order non-linear ODE, one second order linear ODE, and one two-dimensional linear PDE. Using the LS-SVM method as a benchmark, a speed comparison is made for all the problems by timing the training period, and an accuracy comparison is made using the maximum error and mean-squared error on the training and test sets. In general, ToC is shown to be slightly faster (by an order of magnitude or less) and more accurate (by multiple orders of magnitude) over the LS-SVM and CSVM approaches.