Neural Networks to solve Partial Differential Equations: a Comparison with Finite Elements

We compare the Finite Element Method (FEM) simulation of a standard Partial Differential Equation thermal problem of a plate with a hole with a Neural Network (NN) simulation. The largest deviation from the true solution obtained from FEM (0.025 for a solution on the order of unity) is easily achieved with NN too without much tuning of the hyperparameters. A higher accuracy value (0.001) instead requires refinement with an alternative optimizer to reach a similar performance with NN. A rough comparison between the Floating Point Operations values, as a machine-independent quantification of the computational performance, suggests a significant difference between FEM and NN in favour of the former. This also strongly holds for computation time: for an accuracy on the order of $10^{-5}$, FEM and NN require 38 and 1100 seconds, respectively. A detailed analysis of the effect of varying different hyperparameters shows that accuracy and computational time only weakly depend on the major part of them. Accuracies below 0.01 cannot be achieved with the "Adam" optimizers and it looks as though accuracies below $10^{-5}$ cannot be achieved at all. Training turns to be equally effective when performed on points extracted from the FEM mesh.