Practical SVBRDF and shape acquisition using deep dual Imaging
File(s)
Author(s)
Fan, Chongrui
Type
Thesis
Abstract
The appearance of objects can be described by their geometric shapes and spatially varying optical reflectance maps. Traditional methods typically require specialized and complex hardware to recover geometric shapes and underlying physical models while controlling lighting conditions. Although the results are usually accurate, the costly equipment and complex capture process no longer meet the rapidly growing demands of digital content creation. With the advancement of deep learning, many rapid and straightforward methods have emerged, reducing the required number of images to a minimum—sometimes only one. However, these outstanding studies still lag in result quality, especially for structurally complex 3D objects. In this thesis, we focus on using the simplest possible capture devices, such as Off-the-shelf smartphones, and eliminate restrictions on lighting conditions as much as possible. Based on deep neural networks, our methods use two images to predict high-quality shapes and spatially varying reflectance of 3D objects and human faces, finding a sweet spot between quality and capture costs, significantly reducing the user’s capture expenses.
We utilize a smartphone’s multi-lens imaging technology to acquire high-quality shapes and spatially varying reflectance of 3D objects. Our method captures two images simultaneously using the zoom and the wide-angle lens of a smartphone, under both natural illumination and phone flash, working as efficiently as single-shot methods and achieving superior results. Furthermore, specifically for the most popular and intricate 3D object—the human face—we propose a deep learning-based imaging method that predicts the shape and SVBRDF of a subject’s face directly from two images taken from each side of the face, avoiding the need for a pre-learned morphable model and facilitating ease of use. Compared to single-image facial reconstruction methods, our approach excels in rendering faces at extreme angles and provides texture maps that are directly usable in most rendering systems.
We utilize a smartphone’s multi-lens imaging technology to acquire high-quality shapes and spatially varying reflectance of 3D objects. Our method captures two images simultaneously using the zoom and the wide-angle lens of a smartphone, under both natural illumination and phone flash, working as efficiently as single-shot methods and achieving superior results. Furthermore, specifically for the most popular and intricate 3D object—the human face—we propose a deep learning-based imaging method that predicts the shape and SVBRDF of a subject’s face directly from two images taken from each side of the face, avoiding the need for a pre-learned morphable model and facilitating ease of use. Compared to single-image facial reconstruction methods, our approach excels in rendering faces at extreme angles and provides texture maps that are directly usable in most rendering systems.
Version
Open Access
Date Issued
2024-10-04
Date Awarded
01/02/2025
License URL
Advisor
Ghosh, Abhijeet
Publisher Department
Department of Computing
Publisher Institution
Imperial College London
Qualification Level
Doctoral
Qualification Name
Doctor of Philosophy (PhD)