Improving fairness in facial albedo estimation via visual-textual cues
Author(s)
Ren, Xingyu
Deng, Jiankang
Ma, Chao
Yan, Yichao
Yang, Xiaokang
Type
Conference Paper
Abstract
Recent 3D face reconstruction methods have made significant advances in geometry prediction, yet further cosmetic improvements are limited by lagged albedo because inferring albedo from appearance is an ill-posed problem. Although some existing methods consider prior knowledge from illumination to improve albedo estimation, they still produce a light-skin bias due to racially biased albedo models and limited light constraints. In this paper, we reconsider the relationship between albedo and face attributes and propose a ID2Albedo to directly estimate albedo without constraining illumination. Our key insight is that intrinsic semantic attributes such as race, skin color, and age can be used to constrain the albedo map. We first introduce visual-textual cues and design a semantic loss to supervise facial albedo estimation. Specifically, we pre-define text labels such as race, skin color, age, and wrinkles. Then, we employ the text-image model (CLIP) to compute the similarity between the text and the input image, and assign a pseudo-label to each facial image. We constrain generated albedos in the training phase to have the same attributes as the inputs. In addition, we train a high-quality, unbiased facial albedo generator and utilize the semantic loss to learn the mapping from illumination-robust identity features to the albedo latent codes. Finally, our ID2Albedo is trained in a self-supervised way and outperforms state-of-the-art albedo estimation methods in terms of accuracy and fidelity. It is worth mentioning that our approach has excellent generalizability and fairness, especially on in-the- wild data.
Date Issued
2023-08-22
Date Acceptance
2023-06-01
Citation
2023 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2023, pp.4511-4520
ISSN
1063-6919
Publisher
IEEE Computer Society
Start Page
4511
End Page
4520
Journal / Book Title
2023 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)
Copyright Statement
© 2023 IEEE. This CVPR paper is the Open Access version, provided by the Computer Vision Foundation. Except for this watermark, it is identical to the accepted version; the final published version of the proceedings is available on IEEE Xplore.
Source
IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)
Subjects
Computer Science
Computer Science, Artificial Intelligence
Computer Science, Interdisciplinary Applications
Computer Science, Software Engineering
Computer Science, Theory & Methods
Science & Technology
Technology
Publication Status
Published
Start Date
2023-06-17
Finish Date
2023-06-24
Coverage Spatial
Vancouver, Canada