Tensors, which provide a powerful and flexible model for representing multi-attribute data and multi-way interactions, play an indispensable role in modern data science across various fields in science and engineering. A fundamental task is to faithfully recover the tensor from highly incomplete measurements in a statistically and computationally efficient manner. Harnessing the low-rank structure of tensors in the Tucker decomposition, this paper develops a scaled gradient descent (ScaledGD) algorithm to directly recover the tensor factors with tailored spectral initializations, and shows that it provably converges at a linear rate independent of the condition number of the ground truth tensor for two canonical problems -- tensor completion and tensor regression -- as soon as the sample size is above the order of $n^{3/2}$ ignoring other dependencies, where $n$ is the dimension of the tensor. This leads to an extremely scalable approach to low-rank tensor estimation compared with prior art, which suffers from at least one of the following drawbacks: extreme sensitivity to ill-conditioning, high per-iteration costs in terms of memory and computation, or poor sample complexity guarantees. To the best of our knowledge, ScaledGD is the first algorithm that achieves near-optimal statistical and computational complexities simultaneously for low-rank tensor completion with the Tucker decomposition. Our algorithm highlights the power of appropriate preconditioning in accelerating nonconvex statistical estimation, where the iteration-varying preconditioners promote desirable invariance properties of the trajectory with respect to the underlying symmetry in low-rank tensor factorization.
翻译:塔克分解中,塔克分解中,塔克分解器提供了代表多归数据和多路互动的强大和灵活的模型,在科学和工程领域各个领域的现代数据科学中发挥着不可或缺的作用;一项基本任务是以统计和计算效率的方式,忠实地从高度不完整的测量中恢复高压;在塔克分解中,利用高压的低位结构,本文件开发了一种缩放梯度下限算法,以通过量身定制的光谱初始化直接恢复微调系数,并表明它以线性速度趋同,而独立于地面真理回声器的条件数,以两种卡通度问题 -- -- 高超度完成和回落 -- -- 的状态,只要样本规模超过1美元/%3/2美元,就能够忠实地从高度恢复到高度不完全依赖其他情况,这就导致与以往艺术相比,对低压度估算采取极为可伸缩的方法,这至少与以下一种推论:对调的高度敏感度、在接近的卡路里值上,在接近的精确度上,不计价推算中,我们最接近于最接近的不精确的不精确的不精确的推算成本成本,在接近的亚化中,其最接近的不精确的不推算中,其最接近地值成本成本成本成本成本成本在接近于我们的统计推算中,在接近于统计推算中,在接近地的推算中,在接近的推算中,在接近于其最接近于达到最接近和低的推算的推算中,在接近的推算中,或低的推算的推算的推算中,在接近的推算入,即:我们最深的压性压的推算中,在接近于我们最接近于我们最接近的压性推算中,在接近和低的推算的推算中,在接近的推算中,在接近和低的推算的推算中,在接近的推算中,在接近和低的推算中,在接近和低的推算的推算中,或低的推算的推算中,在接近的推算的推算中,即::在接近于我们的推算入,在接近的推算的推算中,在接近的推算的推算中,