High quality neural relighting using practical zonal illumination
File(s)05_sr20241150.pdf (50.42 MB)
Published version
Author(s)
Lin, Arvin
Lin, Yiming
Li, Xiaohui
Ghosh, Abhijeet
Type
Conference Paper
Abstract
We present a method for high-quality image-based relighting using a practical limited zonal illumination field. Our setup can be implemented with commodity components with no dedicated hardware. We employ a set of desktop monitors to illuminate a subject from a near-hemispherical zone and record One-Light-At-A-Time (OLAT) images from multiple viewpoints. We further extrapolate sampling of incident illumination directions beyond the frontal coverage of the monitors by repeating OLAT captures with the subject rotation in relation to the capture setup. Finally, we train our proposed skip-assisted autoencoder and latent diffusion based generative method to learn a high-quality continuous representation of the reflectance function without requiring explicit alignment of the data captured from various viewpoints. This method enables smooth lighting animation for high-frequency reflectance functions and effectively manages to extend incident lighting beyond the practical capture setup's illumination zone. Compared to state-of-the-art methods, our approach achieves superior image-based relighting results, capturing finer skin pore details and extending to passive performance video relighting.
Date Issued
2024-07-03
Date Acceptance
2024-06-07
Citation
Eurographics Symposium on Rendering, 2024
ISBN
978-3-03868-262-2
ISSN
1727-3463
Publisher
The Eurographics Association
Journal / Book Title
Eurographics Symposium on Rendering
Copyright Statement
© 2024 The Authors.
Proceedings published by Eurographics - The European Association for Computer Graphics.
This is an open access article under the terms of the Creative Commons Attribution License, which
permits use, distribution and reproduction in any medium, provided the original work is properly
cited.
Proceedings published by Eurographics - The European Association for Computer Graphics.
This is an open access article under the terms of the Creative Commons Attribution License, which
permits use, distribution and reproduction in any medium, provided the original work is properly
cited.
License URL
Source
Eurographics Symposium on Rendering (EGSR) 2024
Start Date
2024-07-03
Finish Date
2024-07-05
Coverage Spatial
London, UK