Deep neural networks are transforming fields ranging from computer vision to computational medicine, and we recently extended their application to the field of phase-change heat transfer by introducing theory-trained neural networks (TTNs) for a solidification problem \cite{TTN}. Here, we present general, in-depth, and empirical insights into theory-training networks for learning the solution of highly coupled differential equations. We analyze the deteriorating effects of the oscillating loss on the ability of a network to satisfy the equations at the training data points, measured by the final training loss, and on the accuracy of the inferred solution. We introduce a theory-training technique that, by leveraging regularization, eliminates those oscillations, decreases the final training loss, and improves the accuracy of the inferred solution, with no additional computational cost. Then, we present guidelines that allow a systematic search for the network that has the optimal training time and inference accuracy for a given set of equations; following these guidelines can reduce the number of tedious training iterations in that search. Finally, a comparison between theory-training and the rival, conventional method of solving differential equations using discretization attests to the advantages of theory-training not being necessarily limited to high-dimensional sets of equations. The comparison also reveals a limitation of the current theory-training framework that may limit its application in domains where extreme accuracies are necessary.
翻译:深心神经网络正在改变从计算机视野到计算医学的各个领域,我们最近通过引入理论培训神经网络(TTNs),将它们的应用扩大到了阶段变化热传输领域。在这里,我们提出了理论培训神经网络(TTNs),用于固化问题。我们提出了理论培训网络的一般、深入和经验见解,用于学习高度混合差异方程式的解决方案。我们分析了振动性损失对网络在培训数据点满足方程式的能力的恶化影响,根据最终培训损失和推断解决方案的准确性来衡量。我们引入了理论培训技术,通过利用正规化,消除这些振动,减少最终培训损失,提高推断解决方案的准确性,不增加计算成本。然后,我们提出了指导方针,以便系统搜索具有最佳培训时间的网络和特定方程式的推断准确性;遵循这些指导方针,可以减少在搜索中进行重复培训的次数。最后,我们将必要的理论培训与对等模型的对等化加以比较,同时将理论和对等化的对等性理论加以比较,这样可以证明目前不同方程式的对等度的精确性,而这种对等化的对等性也能够证明目前不同方程式的对等性,对等化的对等化的对等化的对等法的对等性,对等性将演示法的对准性将证明,对准性对准性将使得对等性在目前方方方方程式的对准性对准性对准性对准性对准性对准性对准性对准性对准性对准性对准性对准性对准性对准性对准性对准性对准性对准性对准性对准性对准性对准性对准性对准性对准性对准性对准性对准性对准性对准性对准性对准性对准性对性对准性对面性对性对准性对准性对准性对准性对准性对准性对准性对准性对准性对准性对准性对准性对准性对准性对准性对准性对准性对准性对准性对准性对准性对准性对准性对准性对准性对准性对准性对准性对准性对准性对准性对