Spectral methods for nonlinear functionals and functional differential equations

We present a rigorous convergence analysis for cylindrical approximations of nonlinear functionals, functional derivatives, and functional differential equations (FDEs). The purpose of this analysis is twofold: first, we prove that continuous nonlinear functionals, functional derivatives and FDEs can be approximated uniformly on any compact subset of a real Banach space admitting a basis by high-dimensional multivariate functions and high-dimensional partial differential equations (PDEs), respectively. Second, we show that the convergence rate of such functional approximations can be exponential, depending on the regularity of the functional (in particular its Fr\'echet differentiability), and its domain. We also provide necessary and sufficient conditions for consistency, stability and convergence of cylindrical approximations to linear FDEs. These results open the possibility to utilize numerical techniques for high-dimensional systems such as deep neural networks and numerical tensor methods to approximate nonlinear functionals in terms of high-dimensional functions, and compute approximate solutions to FDEs by solving high-dimensional PDEs. Numerical examples are presented and discussed for prototype nonlinear functionals and for an initial value problem involving a linear FDE.