Neural BTF compression and interpolation
File(s)
Accepted version
Author(s)
Rainer, Gilles
Jakob, Wenzel
Ghosh, Abhijeet
Weyrich, Tim
Type
Journal Article
Abstract
The Bidirectional Texture Function (BTF) is a data-driven solution to render materials with complex appearance. A typicalcapture contains tens of thousands of images of a material sample under varying viewing and lighting conditions. While capableof faithfully recording complex light interactions in the material, the main drawback is the massive memory requirement, bothfor storing and rendering, making effective compression of BTF data a critical component in practical applications. Commoncompression schemes used in practice are based on matrix factorization techniques, which preserve the discrete format ofthe original dataset. While this approach generalizes well to different materials, rendering with the compressed dataset stillrelies on interpolating between the closest samples. Depending on the material and the angular resolution of the BTF, thiscan lead to blurring and ghosting artefacts. An alternative approach uses analytic model fitting to approximate the BTF data,using continuous functions that naturally interpolate well, but whose expressive range is often not wide enough to faithfullyrecreate materials with complex non-local lighting effects (subsurface scattering, inter-reflections, shadowing and masking...).In light of these observations, we propose a neural network-based BTF representation inspired by autoencoders: our encodercompresses each texel to a small set of latent coefficients, while our decoder additionally takes in a light and view directionand outputs a single RGB vector at a time. This allows us to continuously query reflectance values in the light and viewhemispheres, eliminating the need for linear interpolation between discrete samples. We train our architecture on fabric BTFswith a challenging appearance and compare to standard PCA as a baseline. We achieve competitive compression ratios andhigh-quality interpolation/extrapolation without blurring or ghosting artifacts.
Date Issued
2019-05-01
Date Acceptance
2019-02-22
Citation
Computer Graphics Forum, 38 (2)
ISSN
0167-7055
Publisher
Wiley
Journal / Book Title
Computer Graphics Forum
Volume
38
Issue
2
Sponsor
Engineering & Physical Science Research Council (EPSRC)
Identifier
http://reality.cs.ucl.ac.uk/projects/btf/rainer19neural.html
Grant Number
EP/N006259/1
Subjects
Software Engineering
0801 Artificial Intelligence and Image Processing
Publication Status
Accepted