We study the problem of estimating the trace of a matrix $A$ that can only be accessed through matrix-vector multiplication. We introduce a new randomized algorithm, Hutch++, which computes a $(1 \pm \epsilon)$ approximation to $tr(A)$ for any positive semidefinite (PSD) $A$ using just $O(1/\epsilon)$ matrix-vector products. This improves on the ubiquitous Hutchinson's estimator, which requires $O(1/\epsilon^2)$ matrix-vector products. Our approach is based on a simple technique for reducing the variance of Hutchinson's estimator using a low-rank approximation step, and is easy to implement and analyze. Moreover, we prove that, up to a logarithmic factor, the complexity of Hutch++ is optimal amongst all matrix-vector query algorithms, even when queries can be chosen adaptively. We show that it significantly outperforms Hutchinson's method in experiments. While our theory mainly requires $A$ to be positive semidefinite, we provide generalized guarantees for general square matrices, and show empirical gains in such applications.
翻译:我们研究的是估算一个只能通过矩阵矢量乘法获得的基质$A的痕迹的问题。 我们引入了一种新的随机算法, Hutch++, 对任何正半无限期(PSD) $A 产品来说, 对任何正半无限期(PSD) $ $ 美元来说, 计算一个( 1 /\ exsilon) 美元近似 $ $tr( A) 美元。 这改进了无处不在的哈钦森测算器, 它需要 $( 1/\ explon2) $ 矩阵矢量产品。 我们的方法基于一种简单的技术, 使用低层次的近似步骤来减少哈钦森测算器的差额, 并且容易执行和分析。 此外, 我们证明, 在一个对数系数上, 赫奇+ 的复杂程度在所有矩阵- 查询算法中是最佳的, 即使在查询可以适应性选择的时候, 。 我们显示它明显地超越了哈钦森在实验中采用的方法。 我们的理论要求以 $A 和 基本实验性 的模型中, 显示这种结果是肯定的。