163
IRUS Total
Downloads
  Altmetric

3D reconstruction of "in-the-wild" faces in images and videos

File Description SizeFormat 
itwmm__tpami_special_issue__compressed.pdfAccepted version765.16 kBAdobe PDFView/Open
Title: 3D reconstruction of "in-the-wild" faces in images and videos
Authors: Booth, J
Roussos, A
Ververas, E
Antonakos, E
Poumpis, S
Panagakis, Y
Zafeiriou, SP
Item Type: Journal Article
Abstract: 3D Morphable Models (3DMMs) are powerful statistical models of 3D facial shape and texture, and among the state-of-the-art methods for reconstructing facial shape from single images. With the advent of new 3D sensors, many 3D facial datasets have been collected containing both neutral as well as expressive faces. However, all datasets are captured under controlled conditions. Thus, even though powerful 3D facial shape models can be learnt from such data, it is difficult to build statistical texture models that are sufficient to reconstruct faces captured in unconstrained conditions ("in-the-wild"). In this paper, we propose the first "in-the-wild" 3DMM by combining a statistical model of facial identity and expression shape with an "in-the-wild" texture model. We show that such an approach allows for the development of a greatly simplified fitting procedure for images and videos, as there is no need to optimise with regards to the illumination parameters. We have collected three new databases that combine "in-the-wild" images and video with ground truth 3D facial geometry, the first of their kind, and report extensive quantitative evaluations using them that demonstrate our method is state-of-the-art.
Issue Date: 1-Nov-2018
Date of Acceptance: 6-Apr-2018
URI: http://hdl.handle.net/10044/1/83711
DOI: 10.1109/TPAMI.2018.2832138
ISSN: 0162-8828
Publisher: Institute of Electrical and Electronics Engineers
Start Page: 2638
End Page: 2652
Journal / Book Title: IEEE Transactions on Pattern Analysis and Machine Intelligence
Volume: 40
Issue: 11
Copyright Statement: © 2018 IEEE. Personal use of this material is permitted. Permission from IEEE must be obtained for all other uses, in any current or future media, including reprinting/republishing this material for advertising or promotional purposes, creating new collective works, for resale or redistribution to servers or lists, or reuse of any copyrighted component of this work in other works.
Sponsor/Funder: Engineering & Physical Science Research Council (E
Funder's Grant Number: EP/N007743/1
Keywords: Algorithms
Databases, Factual
Face
Facial Expression
Female
Humans
Image Processing, Computer-Assisted
Imaging, Three-Dimensional
Machine Learning
Male
Models, Anatomic
Models, Statistical
Pattern Recognition, Automated
Photography
Video Recording
Face
Humans
Imaging, Three-Dimensional
Photography
Facial Expression
Models, Statistical
Algorithms
Models, Anatomic
Image Processing, Computer-Assisted
Video Recording
Databases, Factual
Pattern Recognition, Automated
Female
Male
Machine Learning
Artificial Intelligence & Image Processing
0801 Artificial Intelligence and Image Processing
0806 Information Systems
0906 Electrical and Electronic Engineering
Publication Status: Published
Conference Place: United States
Open Access location: https://ibug.doc.ic.ac.uk/media/uploads/documents/itwmm__tpami_special_issue_.pdf
Online Publication Date: 2018-05-15
Appears in Collections:Computing
Faculty of Engineering