Less is More: Revisiting the Gaussian Mechanism for Differential Privacy

In this paper, we identify that the classic Gaussian mechanism and its variants for differential privacy all suffer from the curse of full-rank covariance matrices, and hence the expected accuracy losses of these mechanisms applied to high dimensional query results, e.g., in $\mathbb{R}^M$, all increase linearly with $M$. To lift this curse, we design a Rank-1 Singular Multivariate Gaussian Mechanism (R1SMG). It achieves $(\epsilon,\delta)$-DP on query results in $\mathbb{R}^M$ by perturbing the results with noise following a singular multivariate Gaussian distribution, whose covariance matrix is a randomly generated rank-1 positive semi-definite matrix. In contrast, the classic Gaussian mechanism and its variants all consider deterministic full-rank covariance matrices. Our idea is motivated by a clue from Dwork et al.'s work on Gaussian mechanism that has been ignored in the literature: when projecting multivariate Gaussian noise with a full-rank covariance matrix onto a set of orthonormal basis in $\mathbb{R}^M$, only the coefficient of a single basis can contribute to the privacy guarantee. This paper makes the following technical contributions. (i) R1SMG achieves $(\epsilon,\delta)$-DP guarantee on query results in $\mathbb{R}^M$, while the magnitude of the additive noise decreases with $M$. Therefore, less is more, i.e., less amount of noise is able to sanitize higher dimensional query results. When $M\rightarrow \infty$, the expected accuracy loss converges to ${2(\Delta_2f)^2}/{\epsilon}$, where $\Delta_2f$ is the $l_2$ sensitivity of the query function $f$. (ii) Compared with other mechanisms, R1SMG is less likely to generate noise with large magnitude that overwhelms the query results, because the kurtosis and skewness of the nondeterministic accuracy loss introduced by R1SMG is larger than that introduced by other mechanisms.