Deep shape and SVBRDF estimation using smartphone multi-lens imaging
File(s)
Accepted version
Author(s)
FAN, Chongrui
Lin, Yiming
Ghosh, Abhijeet
Type
Journal Article
Abstract
We present a deep neural network-based method that acquires high-quality shape and spatially varying reflectance of 3D objects using smartphone multi-lens imaging. Our method acquires two images simultaneously using a zoom lens and a wide angle lens of a smartphone under either natural illumination or phone flash conditions, effectively functioning like a single-shot method. Unlike traditional multi-view stereo methods which require sufficient differences in viewpoint and only estimate depth at a certain coarse scale, our method estimates fine-scale depth by utilising an optical-flow field extracted from subtle baseline and perspective due to different optics in the two images captured simultaneously. We further guide the SVBRDF estimation using the estimated depth, resulting in superior results compared to existing single-shot methods.
Date Issued
2023-10-01
Date Acceptance
2023-08-15
Citation
Computer Graphics Forum: the international journal of the Eurographics Association, 2023, Special issue Pacific Graphics 2023, 42 (7)
ISSN
0167-7055
Publisher
Wiley
Journal / Book Title
Computer Graphics Forum: the international journal of the Eurographics Association
Volume
42
Issue
7
Copyright Statement
Copyright © 2023 Wiley. This is the peer reviewed version of the following article, which has been published in final form at https://onlinelibrary.wiley.com/doi/10.1111/cgf.14972. This article may be used for non-commercial purposes in accordance with Wiley Terms and Conditions for Use of Self-Archived Versions. This article may not be enhanced, enriched or otherwise transformed into a derivative work, without express permission from Wiley or by statutory rights under applicable legislation. Copyright notices must not be removed, obscured or modified. The article must be linked to Wiley’s version of record on Wiley Online Library and any embedding, framing or otherwise making available the article or pages thereof by third parties from platforms, services and websites other than Wiley Online Library must be prohibited.
Edition
Special issue Pacific Graphics 2023
Publication Status
Published
Article Number
ARTN e14972
Date Publish Online
2023-10-30