3DMM-RF: Convolutional radiance fields for 3D face modeling
Author(s)
Galanakis, Stathis
Gecer, Baris
Lattas, Alexandros
Zafeiriou, Stefanos
Type
Conference Paper
Abstract
Facial 3D Morphable Models are a main computer vision subject with countless applications and have been highly optimized in the last two decades. The tremendous improvements of deep generative networks have created various possibilities for improving such models and have attracted wide interest. Moreover, the recent advances in neural radiance fields, are revolutionising novel-view synthesis of known scenes. In this work, we present a facial 3D Morphable Model, which exploits both of the above, and can accurately model a subject’s identity, pose and expression and render it in arbitrary illumination. This is achieved by utilizing a powerful deep style-based generator to overcome two main weaknesses of neural radiance fields, their rigidity and rendering speed. We introduce a style-based generative network that synthesizes in one pass all and only the required rendering samples of a neural radiance field. We create a vast labelled synthetic dataset of facial renders, and train the network, so that it can accurately model and generalize on facial identity, pose and appearance. Finally, we show that this model can accurately be fit to "in-the-wild" facial images of arbitrary pose and illumination, extract the facial characteristics, and be used to re-render the face in controllable conditions.
Date Issued
2023-02-06
Date Acceptance
2023-01-01
Citation
2023 IEEE/CVF Winter Conference on Applications of Computer Vision (WACV), 2023, pp.3525-3536
ISSN
2472-6737
Publisher
IEEE Computer Society
Start Page
3525
End Page
3536
Journal / Book Title
2023 IEEE/CVF Winter Conference on Applications of Computer Vision (WACV)
Copyright Statement
Copyright © 2023, IEEE. This ICCV workshop paper is the Open Access version, provided by the Computer Vision Foundation. Except for this watermark, it is identical to the accepted version; the final published version of the proceedings is available on IEEE Xplore.
Source
23rd IEEE/CVF Winter Conference on Applications of Computer Vision (WACV)
Subjects
Computer Science
Computer Science, Artificial Intelligence
Engineering
Engineering, Electrical & Electronic
Imaging Science & Photographic Technology
Science & Technology
Technology
Publication Status
Published
Start Date
2023-01-03
Finish Date
2023-01-07
Coverage Spatial
Waikoloa, HI, USA