Practical dynamic facial appearance modeling and acquisition
File(s)Practical_Dynamic_Appearance-AuthorVersion-lowres.pdf (4.05 MB)
Accepted version
Author(s)
Gotardo, Paulo
Riviere, Jeremy
Bradley, Derek
Ghosh, A
Beeler, Thabo
Type
Journal Article
Abstract
We present a method to acquire dynamic properties of facial skin appearance, including dynamic diffuse albedo encoding blood flow, dynamic specular intensity, and per-frame high resolution normal maps for a facial performance sequence. The method reconstructs these maps from a purely passive multi-camera setup, without the need for polarization or requiring temporally multiplexed illumination. Hence, it is very well suited for integration with existing passive systems for facial performance capture. To solve this seemingly underconstrained problem, we demonstrate that albedo dynamics during a facial performance can be modeled as a combination of: (1) a static, high-resolution base albedo map, modeling full skin pigmentation; and (2) a dynamic, one-dimensional component in the CIE L*a*b* color space, which explains changes in hemoglobin concentration due to blood flow. We leverage this albedo subspace and additional constraints on appearance and surface geometry to also estimate specular reflection parameters and resolve high-resolution normal maps with unprecedented detail in a passive capture system. These constraints are built into an inverse rendering framework that minimizes the difference of the rendered face to the captured images, incorporating constraints from multiple views for every texel on the face. The presented method is the first system capable of capturing high-quality dynamic appearance maps at full resolution and video framerates, providing a major step forward in the area of facial appearance acquisition.
Date Issued
2018-11-01
Date Acceptance
2018-09-17
Citation
ACM Transactions on Graphics, 2018, 37 (6)
ISSN
0730-0301
Publisher
Association for Computing Machinery
Journal / Book Title
ACM Transactions on Graphics
Volume
37
Issue
6
Copyright Statement
©2018 Copyright held by the owner/author(s). Publication rights licensed to ACM. This is the author's version of the work. It is posted here by permission of ACM for your personal use. Not for redistribution. The definitive version was published in ACM Transactions on Graphics, https://dl.acm.org/citation.cfm?doid=3272127.3275073
Subjects
Science & Technology
Technology
Computer Science, Software Engineering
Computer Science
Inverse Rendering
Passive Capture
Dynamic Appearance
Albedo Subspace
Surface Detail
REFLECTANCE
COLOR
Science & Technology
Technology
Computer Science, Theory & Methods
Computer Science
Inverse Rendering
Passive Capture
Dynamic Appearance
Albedo Subspace
Surface Detail
REFLECTANCE
COLOR
Software Engineering
0801 Artificial Intelligence and Image Processing
0806 Information Systems
Publication Status
Published
Article Number
ARTN 232
Date Publish Online
2018-12-04