OSTeC: one-shot texture completion
File(s)
Author(s)
Gecer, Baris
Deng, Jiankang
Zafeiriou, Stefanos
Type
Conference Paper
Abstract
The last few years have witnessed the great success of non-linear generative models in synthesizing high-quality photorealistic face images. Many recent 3D facial texture reconstruction and pose manipulation from a single image approaches still rely on large and clean face datasets to train image-to-image Generative Adversarial Networks (GANs). Yet the collection of such a large scale high-resolution 3D texture dataset is still very costly and difficult to maintain age/ethnicity balance. Moreover, regression-based approaches suffer from generalization to the in-the-wild conditions and are unable to fine-tune to a target-image. In this work, we propose an unsupervised approach for one-shot 3D facial texture completion that does not re-quire large-scale texture datasets, but rather harnesses the knowledge stored in 2D face generators. The proposed approach rotates an input image in 3D and fill-in the unseen regions by reconstructing the rotated image in a 2D face generator, based on the visible parts. Finally, we stitch the most visible textures at different angles in the UV image-plane. Further, we frontalize the target image by projecting the completed texture into the generator. The qualitative and quantitative experiments demonstrate that the completed UV textures and frontalized images are of high quality, resembles the original identity, can be used to train a texture GAN model for 3DMM fitting and improve pose-invariant face recognition. 1
Date Issued
2021-11-13
Date Acceptance
2021-11-01
Citation
2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2021, pp.7624-7634
Publisher
IEEE
Start Page
7624
End Page
7634
Journal / Book Title
2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)
Copyright Statement
© 2021 IEEE. Personal use of this material is permitted. Permission from IEEE must be obtained for all other uses, in any current or future media, including reprinting/republishing this material for advertising or promotional purposes, creating new collective works, for resale or redistribution to servers or lists, or reuse of any copyrighted component of this work in other works.
Identifier
https://ieeexplore.ieee.org/document/9577745
Source
2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)
Subjects
cs.CV
cs.CV
Publication Status
Published
Start Date
2021-06-20
Finish Date
2021-06-25
Coverage Spatial
Nashville, TN, USA
Date Publish Online
2021-11-13