Conditional Neural Processes~(CNPs) formulate distributions over functions and generate function observations with exact conditional likelihoods. CNPs, however, have limited expressivity for high-dimensional observations, since their predictive distribution is factorized into a product of unconstrained (typically) Gaussian outputs. Previously, this could be handled using latent variables or autoregressive likelihood, but at the expense of intractable training and quadratically increased complexity. Instead, we propose calibrating CNPs with an adversarial training scheme besides regular maximum likelihood estimates. Specifically, we train an energy-based model (EBM) with noise contrastive estimation, which enforces EBM to identify true observations from the generations of CNP. In this way, CNP must generate predictions closer to the ground-truth to fool EBM, instead of merely optimizing with respect to the fixed-form likelihood. From generative function reconstruction to downstream regression and classification tasks, we demonstrate that our method fits mainstream CNP members, showing effectiveness when unconstrained Gaussian likelihood is defined, requiring minimal computation overhead while preserving foundation properties of CNPs.
翻译:有条件的神经过程模型(Conditional Neural Processes,CNPs)用精确条件似然率生成函数分布并产生函数观察结果。然而,对于高维观测,CNPs 的表现力有限,因为它们的预测分布被分解为一组未限制的(通常是)高斯输出。以往的处理方法包括使用潜在变量或自回归似然率,但这样会导致无法进行实用的训练和二次增加的复杂度。相反,我们提出使用对抗训练方法来校准 CNPs,除了常规的最大似然估计外。具体来说,我们用噪声对比估计训练一个能源基础模型(Energy-Based Model,EBM),EBM 必须从 CNP 的生成观察结果中识别出真实观测结果。这样,CNP 必须生成更接近基本真实结果的预测结果,以欺骗 EBM,而不仅仅是优化固定形式的似然率。从生成函数重建到下游的回归和分类任务,我们证明了我们的方法适用于主流的 CNP 成员,在定义了未限制高斯似然率的情况下表现出有效性,同时保留了 CNP 的基本属性,需要最少的计算开销。