Beyond de-identification: an adversarial approach to evaluate the practical guarantees of privacy-preserving systems for collecting and sharing data
File(s)
Author(s)
Gadotti, Andrea
Type
Thesis
Abstract
The growth of data available on individuals, both in quantity and in granularity, has largely determined the inadequacy of traditional de-identification as a model to achieve a good tradeoff between privacy and utility for modern data. Among new models to collect and share data, differential privacy — a theory that formalizes and quantifies privacy using a single parameter 𝜀, called privacy loss — has attracted much attention among researchers, but has proven challenging to apply in practice. To overcome such difficulty, practitioners have mainly opted for two approaches: using differential privacy with a very large privacy loss 𝜀, or avoiding differential privacy altogether and relying on techniques that provide no formal guarantees. In this thesis we empirically evaluate the practical privacy guarantees of two deployed solutions: Apple's Count Mean Sketch — a differentially private mechanism deployed in iOS and MacOS devices to collect data about users' preferences — and Aircloak's Diffix — an interactive system to analyze data, designed to achieve "GDPR-level anonymization". We propose pool inference attacks against local differential privacy and show that they can be used against a popular mechanism to reveal a user's political orientation, while our noise-exploitation attacks can exploit Diffix's mechanism to infer a user's sensitive attribute with high accuracy using a very limited number of queries. Finally, in the last chapter we reflect on the implications of our attacks and of related work. We propose a different paradigm — inspired by the concept of defense in depth from computer security — to achieve a good tradeoff between privacy and utility in the short term, based on a layered approach to reduce the risk of attacks in practice. We then describe OPAL, a data query system for the analysis of mobile phone data that follows such paradigm and was deployed in Senegal and Colombia.
Version
Open Access
Date Issued
2022-08
Date Awarded
2022-12
Copyright Statement
Creative Commons Attribution Licence
License URL
Advisor
de Montjoye, Yves-Alexandre
Publisher Department
Computing
Publisher Institution
Imperial College London
Qualification Level
Doctoral
Qualification Name
Doctor of Philosophy (PhD)