Interpretability and fairness are critical in computer vi-sion and machine learning applications, in particular whendealing with human outcomes, e.g. inviting or not invitingfor a job interview based on application materials that mayinclude photographs. One promising direction to achievefairness is by learning data representations that removethe semantics of protected characteristics, and are there-fore able to mitigate unfair outcomes. All available modelshowever learn latent embeddings which comes at the costof being uninterpretable. We propose to cast this problemas data-to-data translation, i.e. learning a mapping froman input domain to a fair target domain, where a fairnessdefinition is being enforced. Here the data domain can beimages, or any tabular data representation. This task wouldbe straightforward if we had fair target data available, butthis is not the case. To overcome this, we learn a highlyunconstrained mapping by exploiting statistics of residuals– the difference between input data and its translated ver-sion – and the protected characteristics. When applied tothe CelebA dataset of face images with gender attribute asthe protected characteristic, our model enforces equality ofopportunity by adjusting the eyes and lips regions. Intrigu-ingly, on the same dataset we arrive at similar conclusionswhen using semantic attribute representations of images fortranslation. On face images of the recent DiF dataset, withthe same gender attribute, our method adjusts nose regions.In the Adult income dataset, also with protected genderattribute, our model achieves equality of opportunity by,among others, obfuscating the wife and husband relation-ship. Analyzing those systematic changes will allow us toscrutinize the interplay of fairness criterion, chosen pro-tected characteristics, and prediction performance.