Single-shot high-quality facial geometry and skin appearance capture
File(s)
Author(s)
Riviere, Jérémy
Gotardo, Paulo
Bradley, Derek
Ghosh, Abhijeet
Beeler, Thabo
Type
Journal Article
Abstract
We propose a new light-weight face capture system capable of reconstructing both high-quality geometry and detailed appearance maps from a single exposure. Unlike currently employed appearance acquisition systems, the proposed technology does not require active illumination and hence can readily be integrated with passive photogrammetry solutions. These solutions are in widespread use for 3D scanning humans as they can be assembled from off-the-shelf hardware components, but lack the capability of estimating appearance. This paper proposes a solution to overcome this limitation, by adding appearance capture to photogrammetry systems. The only additional hardware requirement to these solutions is that a subset of the cameras are cross-polarized with respect to the illumination, and the remaining cameras are parallel-polarized. The proposed algorithm leverages the images with the two different polarization states to reconstruct the geometry and to recover appearance properties. We do so by means of an inverse rendering framework, which solves per texel diffuse albedo, specular intensity, and high-resolution normals, as well as global specular roughness considering the subsurface scattering nature of skin. We show results for a variety of human subjects of different ages and skin typology, illustrating how the captured fine-detail skin surface and subsurface scattering effects lead to realistic renderings of their digital doubles, also in different illumination conditions.
Date Issued
2020-07-08
Date Acceptance
2020-07-01
Citation
ACM Transactions on Graphics, 2020, 39 (4)
ISSN
0730-0301
Publisher
Association for Computing Machinery (ACM)
Journal / Book Title
ACM Transactions on Graphics
Volume
39
Issue
4
Copyright Statement
© 2020 ACM. This is the author's version of the work. It is posted here by permission of ACM for your personal use. Not for redistribution. The definitive version was published in PUBLICATION, {VOL39, ISS4, (July 2020)} [https://doi.org/10.1145/3386569.3392464]
Subjects
0801 Artificial Intelligence and Image Processing
0806 Information Systems
Software Engineering
Publication Status
Published
Article Number
81
Date Publish Online
2020-07-01