A closed form scale bound for the $(ε, δ)$-differentially private Gaussian Mechanism valid for all privacy regimes

The standard closed form lower bound on $\sigma$ for providing $(\epsilon, \delta)$-differential privacy by adding zero mean Gaussian noise with variance $\sigma^2$ is $\sigma > \Delta\sqrt {2}\epsilon^{-1} \sqrt {\log\left( 5/4\delta^{-1} \right)}$ for $\epsilon \in (0,1)$. We present a similar closed form bound $\sigma \geq \Delta (\sqrt{2}\epsilon)^{-1} \left(\sqrt{z}+\sqrt{z+\epsilon}\right)$ for $z=-\log\left(\delta \left(2-\delta \right)\right)$ that is valid for all $\epsilon > 0$ and is always lower (better) for $\epsilon < 1$ and $\delta \leq 0.946$. Both bounds are based on fulfilling a particular sufficient condition. For $\delta < 1$, we present an analytical bound that is optimal for this condition and is necessarily larger than $\Delta/\sqrt{2\epsilon}$.