Evaluating the Usability of Differential Privacy Tools with Data Practitioners

Differential privacy (DP) has become the gold standard in privacy-preserving data analytics, but implementing it in real-world datasets and systems remains challenging. Recently developed DP tools aim to ease data practitioners' burden in implementing DP solutions, but limited research has investigated these DP tools' usability. Through a usability study with 24 US data practitioners with varying prior DP knowledge, we comprehensively evaluate the usability of four Python-based open-source DP tools: DiffPrivLib, Tumult Analytics, PipelineDP, and OpenDP. Our results suggest that DP tools can help novices learn DP concepts; that Application Programming Interface (API) design and documentation are vital for learnability and error prevention; and that user satisfaction highly correlates with the effectiveness of the tool. We discuss the balance between ease of use and the learning curve needed to appropriately implement DP and also provide recommendations to improve DP tools' usability to broaden adoption.