3D reconstruction of "in-the-wild" faces in images and videos
File(s)
Accepted version
Author(s)
Type
Journal Article
Abstract
3D Morphable Models (3DMMs) are powerful statistical models of 3D facial shape and texture, and among the state-of-the-art methods for reconstructing facial shape from single images. With the advent of new 3D sensors, many 3D facial datasets have been collected containing both neutral as well as expressive faces. However, all datasets are captured under controlled conditions. Thus, even though powerful 3D facial shape models can be learnt from such data, it is difficult to build statistical texture models that are sufficient to reconstruct faces captured in unconstrained conditions ("in-the-wild"). In this paper, we propose the first "in-the-wild" 3DMM by combining a statistical model of facial identity and expression shape with an "in-the-wild" texture model. We show that such an approach allows for the development of a greatly simplified fitting procedure for images and videos, as there is no need to optimise with regards to the illumination parameters. We have collected three new databases that combine "in-the-wild" images and video with ground truth 3D facial geometry, the first of their kind, and report extensive quantitative evaluations using them that demonstrate our method is state-of-the-art.
Date Issued
2018-11-01
Date Acceptance
2018-04-06
Citation
IEEE Transactions on Pattern Analysis and Machine Intelligence, 2018, 40 (11), pp.2638-2652
ISSN
0162-8828
Publisher
Institute of Electrical and Electronics Engineers
Start Page
2638
End Page
2652
Journal / Book Title
IEEE Transactions on Pattern Analysis and Machine Intelligence
Volume
40
Issue
11
Copyright Statement
© 2018 IEEE. Personal use of this material is permitted. Permission from IEEE must be obtained for all other uses, in any current or future media, including reprinting/republishing this material for advertising or promotional purposes, creating new collective works, for resale or redistribution to servers or lists, or reuse of any copyrighted component of this work in other works.
Sponsor
Engineering & Physical Science Research Council (E
Identifier
https://www.ncbi.nlm.nih.gov/pubmed/29993707
Grant Number
EP/N007743/1
Subjects
Algorithms
Databases, Factual
Face
Facial Expression
Female
Humans
Image Processing, Computer-Assisted
Imaging, Three-Dimensional
Machine Learning
Male
Models, Anatomic
Models, Statistical
Pattern Recognition, Automated
Photography
Video Recording
Face
Humans
Imaging, Three-Dimensional
Photography
Facial Expression
Models, Statistical
Algorithms
Models, Anatomic
Image Processing, Computer-Assisted
Video Recording
Databases, Factual
Pattern Recognition, Automated
Female
Male
Machine Learning
Artificial Intelligence & Image Processing
0801 Artificial Intelligence and Image Processing
0806 Information Systems
0906 Electrical and Electronic Engineering
Publication Status
Published
Coverage Spatial
United States
Date Publish Online
2018-05-15