Oja's algorithm of principal component analysis (PCA) has been one of the methods utilized in practice to reduce dimension. In this paper, we focus on the convergence property of the discrete algorithm. To realize that, we view the algorithm as a stochastic process on the parameter space and semi-group. We approximate it by SDEs, and prove large time convergence of the SDEs to ensure its performance. This process is completed in three steps. First, the discrete algorithm can be viewed as a semigroup: $S^k\varphi=\mathbb{E}[\varphi(\mathbf W(k))]$. Second, we construct stochastic differential equations (SDEs) on the Stiefel manifold, i.e. the diffusion approximation, to approximate the semigroup. By proving the weak convergence, we verify that the algorithm is 'close to' the SDEs. Finally, we use the reversibility of the SDEs to prove long-time convergence.
翻译:Oja的主要组件分析算法( PCA) 是实践中用来减少维度的方法之一 。 在本文中, 我们关注离散算法的趋同属性 。 要认识到这一点, 我们将算法视为参数空间和半组的随机过程 。 我们用SDEs 来比较这个算法, 并证明SDEs 有很大的时间趋同以确保它的性能。 这个算法分三步完成 。 首先, 离散算法可以视为一个半组 : $Sk\ varphi ⁇ mathb{E} [\\ varphi( mathbf W(k)$. 。 第二, 我们用SDEs的可逆性来证明长期趋同。