Deep polarization imaging for 3D shape and SVBRDF acquisition
File(s)
Accepted version
Author(s)
Deschaintre, Valentin
Lin, Yiming
Ghosh, Abhijeet
Type
Conference Paper
Abstract
We present a novel method for efficient acquisition of shape and spatially varying reflectance of 3D objects using polarization cues. Unlike previous works that have exploited polarization to estimate material or object appearance under certain constraints (known shape or multiview acquisition), we lift such restrictions by coupling polarization imaging with deep learning to achieve high quality estimate of 3D object shape (surface normals and depth)and SVBRDF using single-view polarization imaging under frontal flash illumination. In addition to acquired polarization images, we provide our deep network with strong novel cues related to shape and reflectance, in the form of a normalized Stokes map and an estimate of diffuse color. We additionally describe modifications to network architecture and training loss which provide further qualitative improvements. We demonstrate our approach to achieve superior results compared to recent works employing deep learning in conjunction with flash illumination.
Date Issued
2021-11-02
Date Acceptance
2021-03-29
Citation
2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2021, pp.15562-15571
Publisher
IEEE
Start Page
15562
End Page
15571
Journal / Book Title
2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)
Copyright Statement
© 2021 IEEE. Personal use of this material is permitted. Permission from IEEE must be obtained for all other uses, in any current or future media, including reprinting/republishing this material for advertising or promotional purposes, creating new collective works, for resale or redistribution to servers or lists, or reuse of any copyrighted component of this work in other works. See http://www.ieee.org/publications_standards/publications/rights/index.html for more information.
Sponsor
Engineering & Physical Science Research Council (EPSRC)
Grant Number
EP/N006259/1
Source
IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)
Publication Status
Published
Start Date
2021-06-19
Finish Date
2021-06-25
Coverage Spatial
Virtual