PCA matrix denoising is uniform

Principal component analysis (PCA) is a simple and popular tool for processing high-dimensional data. We investigate its effectiveness for matrix denoising. We assume i.i.d. high dimensional Gaussian noises with standard deviation $\sigma$ are added to clean data generated from a low dimensional subspace. We show that the distance between each pair of PCA-denoised data point and the clean data point is uniformly bounded by $\Otilde(\sigma)$, assuming a low-rank data matrix with mild singular value assumptions. We show such a condition could arise even if the data lies on curves. We then provide a general lower bound for the error of the denoised data matrix, which indicates PCA denoising gives a uniform error bound that is rate-optimal. Furthermore, we examine how the error bound impacts downstream applications such as empirical risk minimization, clustering, and manifold learning. Numerical results validate our theoretical findings and reveal the importance of the uniform error.