Adaptive Plenoptic Sampling: Theory and Applications

Abstract

Image-Based Rendering (IBR) is an effective technique for rendering novel views of a scene from multi-view images. The plenoptic function enables IBR to be formulated in terms of sampling and reconstruction. In this thesis, we combine the theoretical results from uniform plenoptic sampling with non-uniform camera placement. The central concept is that geometry of the scene can be modelled with a sequence of slanted planes. The positions of the cameras are then derived from the plenoptic spectral analysis of a slanted plane. To this end, we present novel results for the plenoptic spectral analysis of a slanted plane and an algorithm for adaptive plenoptic sampling. The novelty of our spectral analysis lies in the inclusion of two realistic conditions when calculating the plenoptic spectrum: finite scene width and cameras with finite field of view. Using these conditions, we derive an exact closed-form expression for the plenoptic spectrum of a slanted plane with bandlimited texture. From this spectrum, we determine an expression for the maximum spacing between adjacent cameras. Using synthetic and real scenes, we show that this expression is a more accurate gauge of the Nyquist sampling density than the current state-of-the-art. Based on these results, we design an adaptive plenoptic sampling algorithm for a scene with a smoothly varying surface and bandlimited texture. The algorithm operates by determining the best sequence of slanted planes to model the scene given its geometry and a limited number of cameras. Once this sequence of planes is obtained, the algorithm then positions the cameras using our sampling analysis of a slanted plane. Using synthetic and real scenes, we show that this algorithm outperforms uniform sampling. Finally, we also present a novel reconstruction filter for plenoptic sampling that outperforms the state-of-the-art for both synthetic and real scenes. The filter uses interpolators of maximum-order-minimal-support (MOMS).