Nonlinear independent component analysis (nICA) aims at recovering statistically independent latent components that are mixed by unknown nonlinear functions. Central to nICA is the identifiability of the latent components, which had been elusive until very recently. Specifically, Hyv\"arinen et al. have shown that the nonlinearly mixed latent components are identifiable (up to often inconsequential ambiguities) under a generalized contrastive learning (GCL) formulation, given that the latent components are independent conditioned on a certain auxiliary variable. The GCL-based identifiability of nICA is elegant, and establishes interesting connections between nICA and popular unsupervised/self-supervised learning paradigms in representation learning, causal learning, and factor disentanglement. However, existing identifiability analyses of nICA all build upon an unlimited sample assumption and the use of ideal universal function learners -- which creates a non-negligible gap between theory and practice. Closing the gap is a nontrivial challenge, as there is a lack of established ``textbook'' routine for finite sample analysis of such unsupervised problems. This work puts forth a finite-sample identifiability analysis of GCL-based nICA. Our analytical framework judiciously combines the properties of the GCL loss function, statistical generalization analysis, and numerical differentiation. Our framework also takes the learning function's approximation error into consideration, and reveals an intuitive trade-off between the complexity and expressiveness of the employed function learner. Numerical experiments are used to validate the theorems.
翻译:非线性独立组成部分分析(nICA)旨在恢复由未知非线性功能混合的统计独立潜在组成部分。 NICA的核心是潜在组成部分的可识别性,这种潜在组成部分直到最近一直难以找到。具体地说, Hyv\"arinenn et al. 已经表明,非线性混合潜在组成部分在普遍对比学习(GCL)的提法下是可以识别的(往往不相干含混混混)的,因为潜在组成部分以某种辅助变量为独立条件。基于GCL的可识别性是优雅的,在NICA和大众非监督/自我监督的学习模式之间建立了令人感兴趣的联系,直到最近一直难以识别。然而,目前对非线性混合的隐含性潜在组成部分的可识别性分析都建立在无限的样本假设和使用理想的通用函数学习者之间,这在理论和实践之间造成了非隐含性的差距。缩小性差距是一个挑战,因为缺乏固定的Ntribook的常规性分析常规性,在这种未受监督的样本分析、因果性学习的GC的精确性、使用性分析中,我们所使用的核心性的分析性、核心性的分析性的分析功能也反映了了核心性的分析。