We explore the probabilistic partition of unity network (PPOU-Net) model in the context of high-dimensional regression problems. With the PPOU-Nets, the target function for any given input is approximated by a mixture of experts model, where each cluster is associated with a fixed-degree polynomial. The weights of the clusters are determined by a DNN that defines a partition of unity. The weighted average of the polynomials approximates the target function and produces uncertainty quantification naturally. Our training strategy leverages automatic differentiation and the expectation maximization (EM) algorithm. During the training, we (i) apply gradient descent to update the DNN coefficients; (ii) update the polynomial coefficients using weighted least-squares solves; and (iii) compute the variance of each cluster according to a closed-form formula derived from the EM algorithm. The PPOU-Nets consistently outperform the baseline fully-connected neural networks of comparable sizes in numerical experiments of various data dimensions. We also explore the proposed model in applications of quantum computing, where the PPOU-Nets act as surrogate models for cost landscapes associated with variational quantum circuits.
翻译:在高维回归问题的背景下,我们探索了统一网络(PPOU-Net)模型的概率分布模型(PPOU-Net)。在高维回归问题的背景下,任何特定输入的目标函数由混合专家模型近似于任何特定输入的目标函数,其中每个组群都与固定度多元度多元分子相关联。组群的权重由定义统一分区的DNNN确定。多多边网络的加权平均值与目标函数相近,并自然产生不确定性量化。我们的培训战略利用自动差异和预期最大化算法。在培训期间,我们(一)应用梯度下降来更新DNN系数;(二)利用加权最小度最小值的解算法更新多数值;以及(三)根据由EM算法产生的封闭式公式计算每个组群群的偏差。PPOU-Net在各种数据层面的数值实验中,始终超越基线完全连接的神经网络。我们还在量计算应用拟议模型中探索了定量计算模型,其中,PPPOOU-Net的光谱系与量流变制成为基模型。