Recent advances in deep learning have enabled us to address the curse of dimensionality (COD) by solving problems in higher dimensions. A subset of such approaches of addressing the COD has led us to solving high-dimensional PDEs. This has resulted in opening doors to solving a variety of real-world problems ranging from mathematical finance to stochastic control for industrial applications. Although feasible, these deep learning methods are still constrained by training time and memory. Tackling these shortcomings, Tensor Neural Networks (TNN) demonstrate that they can provide significant parameter savings while attaining the same accuracy as compared to the classical Dense Neural Network (DNN). In addition, we also show how TNN can be trained faster than DNN for the same accuracy. Besides TNN, we also introduce Tensor Network Initializer (TNN Init), a weight initialization scheme that leads to faster convergence with smaller variance for an equivalent parameter count as compared to a DNN. We benchmark TNN and TNN Init by applying them to solve the parabolic PDE associated with the Heston model, which is widely used in financial pricing theory.
翻译:最近深层次学习的进展使我们得以通过解决更高层面的问题来解决维度的诅咒(COD)问题。处理COD的一系列方法使我们解决了高层面的PDE。这导致打开了解决各种现实世界问题的大门,从数学融资到工业应用的随机控制等一系列问题。虽然可行,但这些深层次的学习方法仍然受到培训时间和记忆的限制。克服这些缺陷,Tensor神经网络(TNN)表明,它们可以提供显著的参数节约,同时与古典的Dense Neural网络(DNN)达到同样的精确度。此外,我们还表明TNNN可以比DNN更快地进行同样的精确度的培训。除了Tensor网络初始化(TNNN Init)之外,我们还引入了一种重量初始化计划,使等值参数计数与DNN相比出现较小的差异。我们以TNN和TNN Init为基准,通过应用它们来解决与Heston模型相关的parlic PDE,该模型在金融定价理论中广泛使用。