We study the tensor-on-tensor regression, where the goal is to connect tensor responses to tensor covariates with a low Tucker rank parameter tensor/matrix without the prior knowledge of its intrinsic rank. We propose the Riemannian gradient descent (RGD) and Riemannian Gauss-Newton (RGN) methods and cope with the challenge of unknown rank by studying the effect of rank over-parameterization. We provide the first convergence guarantee for the general tensor-on-tensor regression by showing that RGD and RGN respectively converge linearly and quadratically to a statistically optimal estimate in both rank correctly-parameterized and over-parameterized settings. Our theory reveals an intriguing phenomenon: Riemannian optimization methods naturally adapt to over-parameterization without modifications to their implementation. We also prove the statistical-computational gap in scalar-on-tensor regression by a direct low-degree polynomial argument. Our theory demonstrates a "blessing of statistical-computational gap" phenomenon: in a wide range of scenarios in tensor-on-tensor regression for tensors of order three or higher, the computationally required sample size matches what is needed by moderate rank over-parameterization when considering computationally feasible estimators, while there are no such benefits in the matrix settings. This shows moderate rank over-parameterization is essentially "cost-free" in terms of sample size in tensor-on-tensor regression of order three or higher. Finally, we conduct simulation studies to show the advantages of our proposed methods and to corroborate our theoretical findings.
翻译:我们研究强力在强力下回归的方法。 我们的目标是在不事先了解其内在等级的情况下,将强力反应与强力共变同低塔级参数抗拉/矩阵连接起来。 我们提出里曼尼梯度梯度下落(RGD)和里曼尼加乌斯-牛顿(RGN)方法,通过研究超参数等级的影响来应对无名军衔的挑战。 我们通过直度低度中度超度超度回归,为一般抗拉-中度回归提供了第一个趋同保证。 我们的理论显示,RGD和RGN的直线和正方位分向一个统计上的最佳估计。 我们的理论显示,在更低度的极分级模型中, 高比值的模型中位(RGD和RGN), 基本上正确分解和超分数的等级。 我们的理论显示了一种统计- 最佳估计, 高分数级的模型中位模型显示, 当需要的直径直方(Sqron) 或直方(x) 直径直方(x) 直方) 的计算, 直方(Sqor) 直方(x) 模拟) 需要的直方(直方) 直方(直方) 直方) 直方) 直方(直方(直方) 模拟) 模拟) 模拟) 模拟) 模拟) 的计算, 直方(直方(直方(直方(直方) 直方) 直方) 直方(直方(直方(直方) 或直方) 的) 的) (直方) (直方) (直方) (直方) (直方) (直方) (直方) (直方) (直方) (直方) (直方) (直) (直方) (直方) (直方) (直方) (直方) (直方) (直方) (直方) (直方) (直方) (直方) (直方) (直方) (直方) (直方) (直方) (直方) (直方) (直方) (直方) (直方) (直方) (直方) (直方)