Adverse effects of learning algorithms: case studies on data privacy and algorithmic pricing

Abstract

The collection of data at large scale and the deployment of artificial intelligence are reshaping societies around the world. While the positive impact of these changes is often publicised, legitimate questions about their negative impact on individuals are still unanswered. Modern machine learning techniques, for instance, pose new privacy risks yet to be quantified. To investigate these risks, I introduce a new profiling attack against behavioral data. When used against large scale location and transaction datasets, my model accurately identifies a single target within the dataset using only auxiliary information recorded over a disjoint period of time. My results show for the first time how profiling attacks are a re-identification risk at scale, even when preventive mechanisms are used to protect people's privacy.

In addition to data privacy, concerns are growing about intelligent pricing algorithms on digital marketplaces, particularly their potential to impact people's purchasing power by learning collusive strategies. Preliminary studies suggest that collusive behavior between independent pricing algorithms based on reinforcement learning may emerge under certain conditions, but it is difficult to predict how these algorithms will actually behave once deployed in the real world. Here, we introduce an adversarial approach against pricing algorithms, enabling an attacker on a 2-firm iterated market to unilaterally manipulate market prices after having learned how its competitor reacts to price changes. Our results show the necessity to account for adversaries.

Taken together, these results emphasise the importance of regulations aimed at finding a balance between the development of modern technologies and the protection of individuals. Despite the clear potential for society of the rise of big data and artificial intelligence, the risks showcased in this thesis should not be overlooked by researchers and policymakers.