AvatarMe: realistically renderable 3D facial reconstruction "in-the-wild"
File(s)2003.13845v1.pdf (9.26 MB)
Accepted version
Author(s)
Type
Working Paper
Abstract
Over the last years, with the advent of Generative Adversarial Networks
(GANs), many face analysis tasks have accomplished astounding performance, with
applications including, but not limited to, face generation and 3D face
reconstruction from a single "in-the-wild" image. Nevertheless, to the best of
our knowledge, there is no method which can produce high-resolution
photorealistic 3D faces from "in-the-wild" images and this can be attributed to
the: (a) scarcity of available data for training, and (b) lack of robust
methodologies that can successfully be applied on very high-resolution data. In
this paper, we introduce AvatarMe, the first method that is able to reconstruct
photorealistic 3D faces from a single "in-the-wild" image with an increasing
level of detail. To achieve this, we capture a large dataset of facial shape
and reflectance and build on a state-of-the-art 3D texture and shape
reconstruction method and successively refine its results, while generating the
per-pixel diffuse and specular components that are required for realistic
rendering. As we demonstrate in a series of qualitative and quantitative
experiments, AvatarMe outperforms the existing arts by a significant margin and
reconstructs authentic, 4K by 6K-resolution 3D faces from a single
low-resolution image that, for the first time, bridges the uncanny valley.
(GANs), many face analysis tasks have accomplished astounding performance, with
applications including, but not limited to, face generation and 3D face
reconstruction from a single "in-the-wild" image. Nevertheless, to the best of
our knowledge, there is no method which can produce high-resolution
photorealistic 3D faces from "in-the-wild" images and this can be attributed to
the: (a) scarcity of available data for training, and (b) lack of robust
methodologies that can successfully be applied on very high-resolution data. In
this paper, we introduce AvatarMe, the first method that is able to reconstruct
photorealistic 3D faces from a single "in-the-wild" image with an increasing
level of detail. To achieve this, we capture a large dataset of facial shape
and reflectance and build on a state-of-the-art 3D texture and shape
reconstruction method and successively refine its results, while generating the
per-pixel diffuse and specular components that are required for realistic
rendering. As we demonstrate in a series of qualitative and quantitative
experiments, AvatarMe outperforms the existing arts by a significant margin and
reconstructs authentic, 4K by 6K-resolution 3D faces from a single
low-resolution image that, for the first time, bridges the uncanny valley.
Date Issued
2020-03-30
Citation
2020
Publisher
arXiv
Copyright Statement
© 2020 The Author(s)
Identifier
http://arxiv.org/abs/2003.13845v1
Subjects
cs.CV
cs.CV
cs.GR
I.2.10; I.3.7; I.4.1
Notes
Accepted to CVPR2020. Project page: github.com/lattas/AvatarMe with high resolution results, data and more. 10 pages, 9 figures
Publication Status
Published