Computer vision algorithms, e.g. for face recognition, favour groups of
individuals that are better represented in the training data. This happens
because of the generalization that classifiers have to make. It is simpler to
fit the majority groups as this fit is more important to overall error. We
propose to create a balanced training dataset, consisting of the original
dataset plus new data points in which the group memberships are intervened,
minorities become majorities and vice versa. We show that current generative
adversarial networks are a powerful tool for learning these data points, called
contrastive examples. We experiment with the equalized odds bias measure on
tabular data as well as image data (CelebA and Diversity in Faces datasets).
Contrastive examples allow us to expose correlations between group membership
and other seemingly neutral features. Whenever a causal graph is available, we
can put those contrastive examples in the perspective of counterfactuals.