Less is More: Revisiting the Gaussian Mechanism for Differential Privacy

Differential privacy via output perturbation has been a \textit{de facto} standard for releasing query or computation results on sensitive data. However, we identify that all existing Gaussian mechanisms suffer from the curse of full-rank covariance matrices, and hence the expected accuracy losses of these mechanisms equal the trace of the covariance matrix of the noise. To lift this curse, we design a Rank-1 Singular Multivariate Gaussian (R1SMG) mechanism. It achieves $(\epsilon,\delta)$-DP on query results in $\mathbb{R}^M$ by perturbing the results with noise following a singular multivariate Gaussian distribution, whose covariance matrix is a \textbf{randomly} generated rank-1 positive semi-definite matrix. In contrast, the classic Gaussian mechanism and its variants all consider \textbf{deterministic} full-rank covariance matrices. Our idea is motivated by a clue from Dwork et al.'s seminal work on the classic Gaussian mechanism that has been ignored: when projecting multivariate Gaussian noise with a full-rank covariance matrix onto a set of orthonormal basis in $\mathbb{R}^M$, only the coefficient of a single basis can contribute to the privacy guarantee. We make the following contributions. The R1SMG mechanisms achieves $(\epsilon,\delta)$-DP guarantee on query results in $\R^M$, while its expected accuracy loss is lower bounded by $C_R(\Delta_2f)^2$, where $C_R = \frac{2}{\epsilon \psi}$ and $\psi = \Big(\frac{\delta\Gamma(\frac{M-1}{2})}{\sqrt{\pi}\Gamma(\frac{M}{2})}\Big)^{\frac{2}{M-2}}$. We show that $C_R$ has a decreasing trend as $M$ increases, and converges to $\frac{2}{\epsilon}$ as $M$ approaches infinity. Compared with other mechanisms, the R1SMG mechanism is more stable and less likely to generate noise with large magnitude that overwhelms the query results.