Honest-but-curious nets: sensitive attributes of private inputs can be secretly coded into the classifiers' outputs
File(s)
Accepted version
Author(s)
Malekzadeh, Mohammad
Borovykh, Anastasia
Gunduz, Deniz
Type
Conference Paper
Abstract
It is known that deep neural networks, trained for the classification of non-sensitive target attributes, can reveal sensitive attributes of their input data through internal representations extracted by the classifier. We take a step forward and show that deep classifiers can be trained to secretly encode a sensitive attribute of their input data into the classifier's outputs for the target attribute, at inference time. Our proposed attack works even if users have a full white-box view of the classifier, can keep all internal representations hidden, and only release the classifier's estimations for the target attribute. We introduce an information-theoretical formulation for such attacks and present efficient empirical implementations for training honest-but-curious (HBC) classifiers: classifiers that can be accurate in predicting their target attribute, but can also exploit their outputs to secretly encode a sensitive attribute. Our work highlights a vulnerability that can be exploited by malicious machine learning service providers to attack their user's privacy in several seemingly safe scenarios; such as encrypted inferences, computations at the edge, or private knowledge distillation. Experimental results on several attributes in two face-image datasets show that a semi-trusted server can train classifiers that are not only perfectly honest but also accurately curious. We conclude by showing the difficulties in distinguishing between standard and HBC classifiers, discussing challenges in defending against this vulnerability of deep classifiers, and enumerating related open directions for future studies.
Date Issued
2021-11-12
Date Acceptance
2021-09-01
Citation
Proceedings of the 2021 ACM SIGSAC Conference on Computer and Communications Security, 2021, pp.825-844
ISBN
978-1-4503-8454-4
Publisher
ACM
Start Page
825
End Page
844
Journal / Book Title
Proceedings of the 2021 ACM SIGSAC Conference on Computer and Communications Security
Copyright Statement
© 2021 ACM. This is the author's version of the work. It is posted here by permission of ACM for your personal use. Not for redistribution. The definitive version was published in Proceedings of the 2021 ACM SIGSAC Conference on Computer and Communications Security, (Nov 2021) https://dl.acm.org/doi/10.1145/3460120.3484533
Sponsor
Commission of the European Communities
Engineering & Physical Science Research Council (EPSRC)
Identifier
https://mmalekzadeh.github.io/
Grant Number
677854
EP/T023600/1
Source
ACM CCS 2021
Publication Status
Published
Start Date
2021-11-15
Finish Date
2021-11-19
Coverage Spatial
Republic of Korea (Virtual)
Date Publish Online
2021-11-13